Problem Statement

Business Context

Workplace safety in hazardous environments like construction sites and industrial plants is crucial to prevent accidents and injuries. One of the most important safety measures is ensuring workers wear safety helmets, which protect against head injuries from falling objects and machinery. Non-compliance with helmet regulations increases the risk of serious injuries or fatalities, making effective monitoring essential, especially in large-scale operations where manual oversight is prone to errors and inefficiency.

To overcome these challenges, SafeGuard Corp plans to develop an automated image analysis system capable of detecting whether workers are wearing safety helmets. This system will improve safety enforcement, ensuring compliance and reducing the risk of head injuries. By automating helmet monitoring, SafeGuard aims to enhance efficiency, scalability, and accuracy, ultimately fostering a safer work environment while minimizing human error in safety oversight.

Objective

As a data scientist at SafeGuard Corp, you are tasked with developing an image classification model that classifies images into one of two categories:

• With Helmet: Workers wearing safety helmets.
• Without Helmet: Workers not wearing safety helmets.

Data Description

The dataset consists of 631 images, equally divided into two categories:

• With Helmet: 311 images showing workers wearing helmets.
• Without Helmet: 320 images showing workers not wearing helmets.

Dataset Characteristics:

• Variations in Conditions: Images include diverse environments such as construction sites, factories, and industrial settings, with variations in lighting, angles, and worker postures to simulate real-world conditions.
• Worker Activities: Workers are depicted in different actions such as standing, using tools, or moving, ensuring robust model learning for various scenarios.

Installing and Importing the Necessary Libraries

#!pip install tensorflow[and-cuda] numpy==1.25.2 -q
!pip install numpy==1.25.2 pandas==2.0.3 seaborn==0.13.1 tensorflow==2.15.0 scikit-learn==1.2.2 matplotlib==3.7.1 -q

import tensorflow as tf
print("Num GPUs Available:", len(tf.config.list_physical_devices('GPU')))
print(tf.__version__)

Num GPUs Available: 1
2.18.0

Note:

• After running the above cell, kindly restart the notebook kernel (for Jupyter Notebook) or runtime (for Google Colab) and run all cells sequentially from the next cell.

• On executing the above line of code, you might see a warning regarding package dependencies. This error message can be ignored as the above code ensures that all necessary libraries and their dependencies are maintained to successfully execute the code in this notebook.

import os
import random
import numpy as np                                                                               # Importing numpy for Matrix Operations
import pandas as pd
import seaborn as sns
import matplotlib.image as mpimg                                                                              # Importing pandas to read CSV files
import matplotlib.pyplot as plt                                                                  # Importting matplotlib for Plotting and visualizing images
import math                                                                                      # Importing math module to perform mathematical operations
import cv2


# Tensorflow modules
import keras
import tensorflow as tf
from tensorflow.keras.preprocessing.image import ImageDataGenerator                              # Importing the ImageDataGenerator for data augmentation
from tensorflow.keras.models import Sequential                                                   # Importing the sequential module to define a sequential model
from tensorflow.keras.layers import Dense,Dropout,Flatten,Conv2D,MaxPooling2D,BatchNormalization # Defining all the layers to build our CNN Model
from tensorflow.keras.optimizers import Adam,SGD                                                 # Importing the optimizers which can be used in our model
from sklearn import preprocessing                                                                # Importing the preprocessing module to preprocess the data
from sklearn.model_selection import train_test_split                                             # Importing train_test_split function to split the data into train and test
from sklearn.metrics import confusion_matrix
from tensorflow.keras.models import Model
from keras.applications.vgg16 import VGG16                                               # Importing confusion_matrix to plot the confusion matrix

# Display images using OpenCV
from google.colab.patches import cv2_imshow

#Imports functions for evaluating the performance of machine learning models
from sklearn.metrics import confusion_matrix, f1_score,accuracy_score, recall_score, precision_score, classification_report
from sklearn.metrics import mean_squared_error as mse                                                 # Importing cv2_imshow from google.patches to display images

# Ignore warnings
import warnings
warnings.filterwarnings('ignore')

# Set the seed using keras.utils.set_random_seed. This will set:
# 1) `numpy` seed
# 2) backend random seed
# 3) `python` random seed
tf.keras.utils.set_random_seed(812)

Data Overview

Loading the data

#Uncomment and run the following code in case Google Colab is being used
from google.colab import drive
drive.mount('/content/drive')

Mounted at /content/drive

# Load the image file of the dataset
images = np.load('/content/drive/MyDrive/Colab Notebooks/images_proj.npy')

# Load the labels file of the dataset
labels = pd.read_csv('/content/drive/MyDrive/Colab Notebooks/Labels_proj.csv')

Exploratory Data Analysis

Data Overview Let's print the shape of the images and labels

print(images.shape)
print(labels.shape)

(631, 200, 200, 3)
(631, 1)

# Converting images from BGR to RGB
images = [cv2.cvtColor(img, cv2.COLOR_BGR2RGB) for img in images]
plt.imshow(images[5])
print('A sample colored imange with label:{}'.format(labels['Label'][99]))

A sample colored imange with label:1

Observation

• The dataset contains 631 colored images with 200 X 200 pixel.
• The image has three color channel BGR (B=Blue, G=Green, R=Red).
• It has its label, image with or without safety helmet.
• We converted image to BGR to RGB to plot it to matplotlib as it supports RGB.

Plot random images from each of the classes and print their corresponding labels.

A sample image with helmet

plt.imshow(images[99])
print('A sample imange (image no: 99) with label:{}'.format(labels['Label'][99]))

A sample imange (image no: 99) with label:1

A sample image without helmet

plt.imshow(images[361])
print('A sample imange (image no: 361) with label:{}'.format(labels['Label'][361]))

A sample imange (image no: 361) with label:0

Checking for class imbalance

labels['Label'].value_counts()

	count
Label
0	320
1	311

dtype: int64

ax = sns.countplot(x=labels['Label'])
ax.set_xlabel('Label \n No Helmet(0), With Helmet(1)')
ax.set_ylabel('Count')
#ax.set_title('No Helmet(0), With Helmet(1)')
plt.show();

Observation

• Dataset is balanced.
• It has 320 images without helmet (class as 0).
• It has 311 images with helmet (class as 1).

Data Preprocessing

labels_class = {0: 'No Helmet(0)', 1: 'Helmet(1)'}

# This functions takes images and its label and plot them
def plot_images(imges,image_labels, cmap='gray'):
  num_classes=2                                                                  # Number of Classes
  categories=np.unique(image_labels)
  keys=dict(image_labels['Label'])                                                      # Obtaing the unique classes from y_train
  rows = 3                                                                        # Defining number of rows=3
  cols = 4                                                                        # Defining number of columns=4
  fig = plt.figure(figsize=(12, 12))                                               # Defining the figure size to 10x8
  for i in range(cols):
      for j in range(rows):
          random_index = np.random.randint(0, len(image_labels))                        # Generating random indices from the data and plotting the images
          ax = fig.add_subplot(rows, cols, i * rows + j + 1)                      # Adding subplots with 3 rows and 4 columns
          if cmap is not 'gray':
            ax.imshow(imges[random_index])
          else:
            ax.imshow(imges[random_index], cmap)
          ax.set_xlabel(f"Imange No:{random_index}")
          ax.set_title(f"Label:" + labels_class[keys[random_index]])
  plt.show()

Images before pre-processing steps

plt.imshow(images[5]) # Display a grayscale image
plt.title('Colored Image')
plt.show()

plot_images(images,labels)

Observation

• 12 random colored images has been plotted with their labels.

Converting images to grayscale

def Conver2gray(imges, height=None, width=None):
  imges_gray = []
  for img in imges:
    gray_img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    # adding RGB channel in the gray image to run vgg
    gray_img_rgb = cv2.merge([gray_img, gray_img, gray_img])
    #print(f'shape:{gray_img_rgb.shape}')
    imges_gray.append(gray_img_rgb)

  if height is not None and width is not None:
    imges_gray = resize_images(imges_gray, width, height)


  return np.array(imges_gray)

# Converting image to grayscale
print(f'image shape before converting to grayscale: {np.array(images).shape}')
images_gray = Conver2gray(images)
print(f'image shape after converting to grayscale: {images_gray.shape}')

plt.imshow(images_gray[5], cmap='gray') # Display a grayscale image
plt.title('Grayscale Image')
plt.show()

image shape before converting to grayscale: (631, 200, 200, 3)
image shape after converting to grayscale: (631, 200, 200, 3)

plot_images(images_gray,labels, cmap='gray')

Observation

• Gray images has been plotted after converting the images to grayscale

Resizing Image

def resize_images(images, width = 64, height=64):
  images_resized = []
  for i in range(len(images)):
    images_resized.append( cv2.resize(images[i], (width, height), interpolation=cv2.INTER_LINEAR))
  resize_image = np.array(images_resized)
  print(f'image shape after rescaling: {resize_image.shape}')
  return resize_image

# We resized image because of lack of GPU to run model in colab
IMAGE_WIDTH=64
IMAGE_HEIGHT=64
IMAGE_CHANNEL=3

dropout_rate = .5

images = resize_images(images, IMAGE_WIDTH, IMAGE_HEIGHT)
images_gray = resize_images(images_gray, IMAGE_WIDTH, IMAGE_HEIGHT)

image shape after rescaling: (631, 64, 64, 3)
image shape after rescaling: (631, 64, 64, 3)

Images after data preprocessing

Colored Picture

plot_images(images,labels)

Gray colored picture

plot_images(images_gray,labels)

Observation

• We resize the image dataset to 64 X 64 color (3 channels with RGB) pixel for our model. The downscaling image size impact image quality.
• VGG 16 supports max 220 X 220 pixel and min 32 X 32 pixel. We downscaled our dataset image size to 64 X 64 to run our model with Google Colab free GPU.

Splitting the dataset

from sklearn.model_selection import train_test_split

def get_training_validation_testing_data(images, labels):
  X_train, X_temp, y_train, y_temp = train_test_split(np.array(images),labels , test_size=0.30, random_state=42,stratify=labels)
  X_val, X_test, y_val, y_test = train_test_split(X_temp,y_temp , test_size=0.5, random_state=42,stratify=y_temp)

  print("Training:")
  print(X_train.shape, y_train.shape)
  print("Validation:")
  print(X_val.shape, y_val.shape)
  print("Testing:")
  print(X_test.shape, y_test.shape)

  classes = ['No Helmet 0', 'Helmet 1']
  def get_label_stat(y_label, ds_type):
      y_label = np.array(y_label).flatten()
      for i in range (len(classes)):
          #print(np.array(y_label))
          total = sum(y_label == i)
          #print(total)
          print("After Splitting, {} data set has counts of Helmet = {}: {} with probability {:.2f}%".format(ds_type, i, total, total/len(y_label) * 100))


  get_label_stat(y_train, "Training")
  get_label_stat(y_val, "Validation")
  get_label_stat(y_test, "Testing")

  return X_train, X_val, X_test, y_train, y_val, y_test

print("Spliting dataset to training, validataion, and testing for Color images")
X_train, X_val, X_test, y_train, y_val, y_test = get_training_validation_testing_data(images, labels)

Spliting dataset to training, validataion, and testing for Color images
Training:
(441, 64, 64, 3) (441, 1)
Validation:
(95, 64, 64, 3) (95, 1)
Testing:
(95, 64, 64, 3) (95, 1)
After Splitting, Training data set has counts of Helmet = 0: 224 with probability 50.79%
After Splitting, Training data set has counts of Helmet = 1: 217 with probability 49.21%
After Splitting, Validation data set has counts of Helmet = 0: 48 with probability 50.53%
After Splitting, Validation data set has counts of Helmet = 1: 47 with probability 49.47%
After Splitting, Testing data set has counts of Helmet = 0: 48 with probability 50.53%
After Splitting, Testing data set has counts of Helmet = 1: 47 with probability 49.47%

print("Spliting dataset to training, validataion, and testing for gray scaled images")
X_train_gray, X_val_gray, X_test_gray, y_train_gray, y_val_gray, y_test_gray = get_training_validation_testing_data(images_gray, labels)

Spliting dataset to training, validataion, and testing for gray scaled images
Training:
(441, 64, 64, 3) (441, 1)
Validation:
(95, 64, 64, 3) (95, 1)
Testing:
(95, 64, 64, 3) (95, 1)
After Splitting, Training data set has counts of Helmet = 0: 224 with probability 50.79%
After Splitting, Training data set has counts of Helmet = 1: 217 with probability 49.21%
After Splitting, Validation data set has counts of Helmet = 0: 48 with probability 50.53%
After Splitting, Validation data set has counts of Helmet = 1: 47 with probability 49.47%
After Splitting, Testing data set has counts of Helmet = 0: 48 with probability 50.53%
After Splitting, Testing data set has counts of Helmet = 1: 47 with probability 49.47%

Ovservation

• We split the dataset into training, validation, and testing.
• All dataset split has almost the same distributions of the labels.
• Label of image dataset:
  • 0 represents Image doesn't have a safety helmet.
  • 1 represents Image has safety helmet.

Data Normalization

Since the image pixel values range from 0-255, our method of normalization here will be scaling - we shall divide all the pixel values by 255 to standardize the images to have values between 0-1.

# Normalizing the Color image pixels
X_train_normalized = X_train.astype('float32')/255.0
X_val_normalized = X_val.astype('float32')/255.0
X_test_normalized = X_test.astype('float32')/255.0

# Normalizing the Gray image pixels
X_train_normalized_gray = X_train_gray.astype('float32')/255.0
X_val_normalized_gray = X_val_gray.astype('float32')/255.0
X_test_normalized_gray = X_test_gray.astype('float32')/255.0

Model Building

Criteria

• Convolution (CNN) layers with filtering, max pooling, and artificial neural network with 3 hidden layer:

  • RELU activation function in CNN, and hidden layer.
  • Drop out - .5 has been applied for few model to compare
  • Adam optimizer
  • batch size = 64, epoc/number of iteration = 10
  • Sigmoid output layer

• VGG-16 model as transfer learning

  • VGG-16 model with freezing its convolution layer, and a sigmoid output layer.
  • VGG-16 convolution layer, and a ANN with a sigmoid output layer.
  • VGG-16 convolution layer with audmentation, and a ANN with a sigmoid output layer.
  • RELU activation function in hidden layer.
  • Drop out - .5 has been applied for few model to compare
  • Adam optimizer
  • batch size = 64, epoc/number of iteration = 10
  • Sigmoid output layer

• We will build models for each of the criteria with both colored and gray images.

Model Evaluation Criterion

• The model successful predictions as:

  • classifying an image with safety helmet and indeed the image originaly has a helmet. This is known as True Positive (TP).
  • classifying an image without safety helmet and indeed the image originaly doesn't have a helmet. This is known as True Negative (TN).

• The model can make wrong predictions as:

  • classifying an image with safety helmet. However, the image originaly doesn't have a helmet. This is known as False Positive (FP).
  • classifying an image without safety helmet. However the image originaly has a helmet. This is known as False Negative (FN).

• Which case is more important?

  • The main object is to build a system to classify, safety helment of workers, therefore, FP is important as our objective will be to minimized it as much as possible. Hence, We will use Recall to evaluate our models.

• Model evaluation metrics definition:

  • Popular machine learning performance evaluation metrics are:
    • Accuracy, Precision, Recall, and F1-score.

# results_eval has the cummulative model evaluation metrics
columns_eval_metric = ["model", "Train_Accuracy","Validation_Accuracy", "Train_Recall", "Validation_Recall", "Train_Precission", "Validation_Precission", "Train_F1","Validation_F1"]
results_eval = pd.DataFrame(columns=columns_eval_metric)
results_eval

	model	Train_Accuracy	Validation_Accuracy	Train_Recall	Validation_Recall	Train_Precission	Validation_Precission	Train_F1	Validation_F1

Utility Functions

# defining a function to compute different metrics to check performance of a classification model built using statsmodels
def model_performance_classification(model, predictors, target):
    """
    Function to compute different metrics to check classification model performance

    model: classifier
    predictors: independent variables
    target: dependent variable
    """

    # checking which probabilities are greater than threshold
    pred = model.predict(predictors).reshape(-1)>0.5

    target = target.to_numpy().reshape(-1)


    acc = accuracy_score(target, pred)  # to compute Accuracy
    recall = recall_score(target, pred, average='weighted')  # to compute Recall
    precision = precision_score(target, pred, average='weighted')  # to compute Precision
    f1 = f1_score(target, pred, average='weighted')  # to compute F1-score

    # creating a dataframe of metrics
    df_perf = pd.DataFrame({"Accuracy": acc, "Recall": recall, "Precision": precision, "F1 Score": f1,},index=[0],)

    return df_perf

# This function build and return confusion matrix
def get_confusion_matrix(model,predictors,target,  ml=False):
    """
    Function to plot the confusion matrix

    model: classifier
    predictors: independent variables
    target: dependent variable
    ml: To specify if the model used is an sklearn ML model or not (True means ML model)
    """

    # checking which probabilities are greater than threshold
    pred = model.predict(predictors).reshape(-1)>0.5

    target = target.to_numpy().reshape(-1)

    # Plotting the Confusion Matrix using confusion matrix() function which is also predefined tensorflow module
    confusion_matrix = tf.math.confusion_matrix(target,pred)
    return confusion_matrix

def plot(history, names):
    """
    Function to plot loss/accuracy

    history: an object which stores the metrics and losses.
    name: can be one of Loss or Accuracy
    """
    i = 0
    fig, ax = plt.subplots(1, len(names), figsize=(20, 6)) #Creating a subplot with figure and axes.
    for name in names:
      ax[i].plot(history.history[name]) #Plotting the train accuracy or train loss
      ax[i].plot(history.history['val_'+name]) #Plotting the validation accuracy or validation loss

      ax[i].set_title('Model ' + name.capitalize()) #Defining the title of the plot.
      ax[i].set_ylabel(name.capitalize()) #Capitalizing the first letter.
      ax[i].set_xlabel('Epoch') #Defining the label for the x-axis.
      ax[i].legend(['Train', 'Validation']) #Defining the legend, loc controls the position of the legend.
      i += 1
    plt.tight_layout()
    plt.show()

#plot(history,['loss','accuracy', 'recall'])
#plot(history,['loss','accuracy'])

def show_performance_metrics(history, model_name, model, X_training, y_training, X_validation, y_validation, is_testing = False):
  """
    Function to show performance metrics

    history: an object which stores the metrics and losses.
    model_name: name of the model
    model: model object
    X_training: training data
    y_training: training labels
    X_validation: validation data
    y_validation: validation labels
    is_testing: flag for testing data
    """
  # Plotting loss and accuracy of model
  if is_testing == False:
    plot(history,['loss','accuracy'])

  test_validation = "Test" if is_testing == True else "Validation"

  # get model performance metrics
  model_train_perf = model_performance_classification(model, X_training, y_training)
  model_valid_perf = model_performance_classification(model, X_validation, y_validation)

  # get confusion metrics of training and validation/testing and draw them horizontally
  cm1 = get_confusion_matrix(model, X_training, y_training)
  cm2 = get_confusion_matrix(model, X_validation, y_validation)

  fig, ax =plt.subplots(1,2, figsize=(12, 5))
  classes = ['No Helmet 0', 'Helmet 1']

  sns.heatmap(cm1, xticklabels=classes, yticklabels=classes, annot=True, fmt="", ax=ax[0], cmap="coolwarm")
  ax[0].set_title("Training")

  sns.heatmap(cm2, xticklabels=classes, yticklabels=classes, annot=True, fmt="", ax=ax[1], cmap="winter")
  ax[1].set_title(test_validation)
  plt.show()

  print("Classification Report")
  print(classification_report(y_validation, model.predict(X_validation) > 0.5, target_names=classes))



  model_train_perf_dict = dict((k, v[0]) for k, v in model_train_perf.to_dict(orient='list').items())
  model_valid_perf_dict = dict((k, v[0]) for k, v in model_valid_perf.to_dict(orient='list').items())

  index = results_eval.shape[0]

  #columns_eval_metric = ["model", "Train_Accuracy","Validation_Accuracy", "Train_Recall", "Validation_Recall", "Train_Precission", "Validation_Precission", "Train_F1","Validation_F1"]
  results_eval.loc[index] = [model_name] + [model_train_perf_dict["Accuracy"], model_valid_perf_dict["Accuracy"], model_train_perf_dict["Recall"], model_valid_perf_dict["Recall"], model_train_perf_dict["Precision"], model_valid_perf_dict["Precision"], model_train_perf_dict["F1 Score"], model_valid_perf_dict["F1 Score"]]

  # results_eval is the cummulative model evaluation dataframe
  return results_eval

def get_binary_output_from_probability(prod):
  return 1 if prod > 0.5 else 0

def show_visulization(X, X_norm, y, model, height, width, channel, index=[0, -1]):
  """
  Function to show images and its prediction with true label.
  X: Image matrix
  X_norm: Normalized Image matrix
  y: true label of image
  mode: model
  height: height of the image used in the model
  width: width of the image used in the model
  channel: channel of the image used in the model
  index: index of the images to be plotted
  """
  for i in index:
    plt.figure(figsize=(2,2))
    plt.imshow(X[i])
    plt.show()
    prob = model.predict(X_norm[i].reshape(1,height,width,channel))[0][0]
    output = get_binary_output_from_probability(prob)
    print(f'Predicted Probability: {prob}, and  Label {output}')
    print('True Label:', np.array(y)[i][0])

Model 1: Simple Convolutional Neural Network (CNN)

def get_cnn_color_model(drop_out_rate= None):
  """
  Function to get cnn model
  drop_out_rate: dropout rate to be applied
  """

  # Initializing Model
  model_1 = Sequential()

  if drop_out_rate is not None:
    drop_out = Dropout(rate=drop_out_rate)


  # 3 Convolutional layers with 64 (3 X 3) filters with max pooling (2 X 2)
  model_1.add(Conv2D(64, (3, 3), activation='relu', padding="same", input_shape=(IMAGE_WIDTH, IMAGE_HEIGHT, IMAGE_CHANNEL))) #Complete the code to define the shape of the input image
  model_1.add(MaxPooling2D((2, 2), padding='same'))

  if drop_out_rate is not None:
    model_1.add(drop_out)

  model_1.add(Conv2D(64, (3,3), activation='relu', padding="same")) #Complete the code to define the number of output channels,the kernel shape and the activation function
  model_1.add(MaxPooling2D((2,2), padding='same')) #Complete the code to define the shape of the pooling kernel

  if drop_out_rate is not None:
    model_1.add(drop_out)

  model_1.add(Conv2D(64, (3,3), activation='relu', padding="same")) #Complete the code to define the number of output channels,the kernel shape and the activation function
  model_1.add(MaxPooling2D((2,2), padding='same')) #Complete the code to define the shape of the pooling kernel


  # Flatten and Dense layers
  model_1.add(Flatten())
  model_1.add(Dense(256, activation='relu', kernel_initializer='he_uniform'))
  if drop_out_rate is not None:
    model_1.add(drop_out)

  model_1.add(Dense(128, activation='relu', kernel_initializer='he_uniform'))  #Complete the code to define the number of neurons in the output layer and the activation function
  if drop_out_rate is not None:
    model_1.add(drop_out)

  model_1.add(Dense(64, activation='relu', kernel_initializer='he_uniform'))

  # Adding a dense output layer
  model_1.add(Dense(1, activation='sigmoid'))


  # Compile with Adam Optimizer
  opt = Adam(learning_rate=.001) #Complete the code to define the learning rate.
  model_1.compile(optimizer=opt, loss='binary_crossentropy', metrics=["accuracy","recall"]) #Complete the code to define the metric of choice from Precision,f1_score,Recall

  # Summary
  model_1.summary()

  tf.config.run_functions_eagerly(True)

  history_cnn = model_1.fit(
              X_train_normalized, np.array(y_train),
              epochs=10,
              validation_data=(X_val_normalized, np.array(y_val)),
              batch_size=64,
              shuffle=True,
              verbose=2
  )

  return history_cnn, model_1

def get_cnn_gray_model(drop_out_rate= None):
  """
  Function to get cnn model
  drop_out_rate: dropout rate to be applied
  """

  # Initializing Model
  model_1 = Sequential()

  if drop_out_rate is not None:
    drop_out = Dropout(rate=drop_out_rate)


  # 3 Convolutional layers with 64 (3 X 3) filters with max pooling (2 X 2)
  model_1.add(Conv2D(64, (3, 3), activation='relu', padding="same", input_shape=(IMAGE_WIDTH, IMAGE_HEIGHT, IMAGE_CHANNEL))) #Complete the code to define the shape of the input image
  model_1.add(MaxPooling2D((2, 2), padding='same'))

  if drop_out_rate is not None:
    model_1.add(drop_out)

  model_1.add(Conv2D(64, (3,3), activation='relu', padding="same")) #Complete the code to define the number of output channels,the kernel shape and the activation function
  model_1.add(MaxPooling2D((2,2), padding='same')) #Complete the code to define the shape of the pooling kernel

  if drop_out_rate is not None:
    model_1.add(drop_out)

  model_1.add(Conv2D(64, (3,3), activation='relu', padding="same")) #Complete the code to define the number of output channels,the kernel shape and the activation function
  model_1.add(MaxPooling2D((2,2), padding='same')) #Complete the code to define the shape of the pooling kernel


  # Flatten and Dense layers
  model_1.add(Flatten())
  model_1.add(Dense(256, activation='relu', kernel_initializer='he_uniform'))
  if drop_out_rate is not None:
    model_1.add(drop_out)

  model_1.add(Dense(128, activation='relu', kernel_initializer='he_uniform'))  #Complete the code to define the number of neurons in the output layer and the activation function
  if drop_out_rate is not None:
    model_1.add(drop_out)

  model_1.add(Dense(64, activation='relu', kernel_initializer='he_uniform'))

  # Adding a dense output layer
  model_1.add(Dense(1, activation='sigmoid'))


  # Compile with Adam Optimizer
  opt = Adam(learning_rate=.001) #Complete the code to define the learning rate.
  model_1.compile(optimizer=opt, loss='binary_crossentropy', metrics=["accuracy","recall"]) #Complete the code to define the metric of choice from Precision,f1_score,Recall

  # Summary
  model_1.summary()

  tf.config.run_functions_eagerly(True)

  history_cnn = model_1.fit(
              X_train_normalized_gray, np.array(y_train_gray),
              epochs=10,
              validation_data=(X_val_normalized_gray, np.array(y_val_gray)),
              batch_size=64,
              shuffle=True,
              verbose=2
  )

  return history_cnn, model_1

def get_cnn_model(drop_out_rate= None):
  history_cnn, model_1 = get_cnn_color_model(drop_out_rate)
  history_cnn_gray, model_1_gray = get_cnn_gray_model(drop_out_rate)
  return history_cnn, model_1, history_cnn_gray, model_1_gray

nested_dict = {}
history_cnn, model_1, history_cnn_gray, model_1_gray = get_cnn_model()

Model: "sequential_14"

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ conv2d_18 (Conv2D)              │ (None, 64, 64, 64)     │         1,792 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ max_pooling2d_18 (MaxPooling2D) │ (None, 32, 32, 64)     │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ conv2d_19 (Conv2D)              │ (None, 32, 32, 64)     │        36,928 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ max_pooling2d_19 (MaxPooling2D) │ (None, 16, 16, 64)     │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ conv2d_20 (Conv2D)              │ (None, 16, 16, 64)     │        36,928 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ max_pooling2d_20 (MaxPooling2D) │ (None, 8, 8, 64)       │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ flatten_14 (Flatten)            │ (None, 4096)           │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_44 (Dense)                │ (None, 256)            │     1,048,832 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_45 (Dense)                │ (None, 128)            │        32,896 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_46 (Dense)                │ (None, 64)             │         8,256 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_47 (Dense)                │ (None, 1)              │            65 │
└─────────────────────────────────┴────────────────────────┴───────────────┘

 Total params: 1,165,697 (4.45 MB)

 Trainable params: 1,165,697 (4.45 MB)

 Non-trainable params: 0 (0.00 B)

Epoch 1/10
7/7 - 1s - 129ms/step - accuracy: 0.6417 - loss: 0.6300 - recall: 0.5622 - val_accuracy: 0.8737 - val_loss: 0.4037 - val_recall: 0.9787
Epoch 2/10
7/7 - 1s - 114ms/step - accuracy: 0.9252 - loss: 0.2478 - recall: 0.9263 - val_accuracy: 0.9789 - val_loss: 0.1386 - val_recall: 0.9787
Epoch 3/10
7/7 - 1s - 115ms/step - accuracy: 0.9433 - loss: 0.1403 - recall: 0.9401 - val_accuracy: 0.8211 - val_loss: 0.5138 - val_recall: 1.0000
Epoch 4/10
7/7 - 1s - 113ms/step - accuracy: 0.9478 - loss: 0.1316 - recall: 0.9585 - val_accuracy: 0.9579 - val_loss: 0.1666 - val_recall: 0.9787
Epoch 5/10
7/7 - 1s - 117ms/step - accuracy: 0.9773 - loss: 0.0566 - recall: 0.9908 - val_accuracy: 0.9789 - val_loss: 0.1297 - val_recall: 0.9787
Epoch 6/10
7/7 - 1s - 120ms/step - accuracy: 0.9819 - loss: 0.0520 - recall: 0.9816 - val_accuracy: 0.9789 - val_loss: 0.1190 - val_recall: 0.9787
Epoch 7/10
7/7 - 1s - 119ms/step - accuracy: 0.9887 - loss: 0.0338 - recall: 0.9816 - val_accuracy: 0.9789 - val_loss: 0.1536 - val_recall: 1.0000
Epoch 8/10
7/7 - 1s - 122ms/step - accuracy: 0.9932 - loss: 0.0140 - recall: 0.9954 - val_accuracy: 0.9789 - val_loss: 0.1349 - val_recall: 0.9787
Epoch 9/10
7/7 - 1s - 116ms/step - accuracy: 1.0000 - loss: 0.0068 - recall: 1.0000 - val_accuracy: 0.9895 - val_loss: 0.1487 - val_recall: 1.0000
Epoch 10/10
7/7 - 1s - 116ms/step - accuracy: 0.9977 - loss: 0.0052 - recall: 0.9954 - val_accuracy: 0.9789 - val_loss: 0.1696 - val_recall: 1.0000

Model: "sequential_15"

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ conv2d_21 (Conv2D)              │ (None, 64, 64, 64)     │         1,792 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ max_pooling2d_21 (MaxPooling2D) │ (None, 32, 32, 64)     │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ conv2d_22 (Conv2D)              │ (None, 32, 32, 64)     │        36,928 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ max_pooling2d_22 (MaxPooling2D) │ (None, 16, 16, 64)     │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ conv2d_23 (Conv2D)              │ (None, 16, 16, 64)     │        36,928 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ max_pooling2d_23 (MaxPooling2D) │ (None, 8, 8, 64)       │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ flatten_15 (Flatten)            │ (None, 4096)           │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_48 (Dense)                │ (None, 256)            │     1,048,832 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_49 (Dense)                │ (None, 128)            │        32,896 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_50 (Dense)                │ (None, 64)             │         8,256 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_51 (Dense)                │ (None, 1)              │            65 │
└─────────────────────────────────┴────────────────────────┴───────────────┘

 Total params: 1,165,697 (4.45 MB)

 Trainable params: 1,165,697 (4.45 MB)

 Non-trainable params: 0 (0.00 B)

Epoch 1/10
7/7 - 1s - 127ms/step - accuracy: 0.6259 - loss: 0.6471 - recall: 0.6820 - val_accuracy: 0.5474 - val_loss: 0.6389 - val_recall: 0.0851
Epoch 2/10
7/7 - 1s - 116ms/step - accuracy: 0.8209 - loss: 0.4571 - recall: 0.7512 - val_accuracy: 0.9895 - val_loss: 0.2235 - val_recall: 1.0000
Epoch 3/10
7/7 - 1s - 115ms/step - accuracy: 0.9751 - loss: 0.1230 - recall: 0.9724 - val_accuracy: 0.9789 - val_loss: 0.1339 - val_recall: 0.9787
Epoch 4/10
7/7 - 1s - 114ms/step - accuracy: 0.9841 - loss: 0.0339 - recall: 0.9816 - val_accuracy: 0.9789 - val_loss: 0.1893 - val_recall: 0.9787
Epoch 5/10
7/7 - 1s - 115ms/step - accuracy: 0.9841 - loss: 0.0301 - recall: 0.9770 - val_accuracy: 0.6842 - val_loss: 1.7011 - val_recall: 1.0000
Epoch 6/10
7/7 - 1s - 116ms/step - accuracy: 0.8957 - loss: 0.5022 - recall: 0.8848 - val_accuracy: 0.8421 - val_loss: 0.5047 - val_recall: 1.0000
Epoch 7/10
7/7 - 1s - 115ms/step - accuracy: 0.9592 - loss: 0.1132 - recall: 0.9954 - val_accuracy: 0.9474 - val_loss: 0.1704 - val_recall: 0.9149
Epoch 8/10
7/7 - 1s - 114ms/step - accuracy: 0.9728 - loss: 0.0833 - recall: 0.9493 - val_accuracy: 0.9895 - val_loss: 0.1806 - val_recall: 1.0000
Epoch 9/10
7/7 - 1s - 114ms/step - accuracy: 0.9955 - loss: 0.0326 - recall: 1.0000 - val_accuracy: 0.9895 - val_loss: 0.1403 - val_recall: 1.0000
Epoch 10/10
7/7 - 1s - 115ms/step - accuracy: 0.9887 - loss: 0.0325 - recall: 0.9816 - val_accuracy: 0.9895 - val_loss: 0.1835 - val_recall: 1.0000

show_performance_metrics(history_cnn, 'CNN', model_1, X_train_normalized, y_train, X_val_normalized, y_val)

14/14 ━━━━━━━━━━━━━━━━━━━━ 0s 13ms/step
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 20ms/step
14/14 ━━━━━━━━━━━━━━━━━━━━ 0s 13ms/step
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 19ms/step

Classification Report
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 18ms/step
              precision    recall  f1-score   support

 No Helmet 0       1.00      0.96      0.98        48
    Helmet 1       0.96      1.00      0.98        47

    accuracy                           0.98        95
   macro avg       0.98      0.98      0.98        95
weighted avg       0.98      0.98      0.98        95

	model	Train_Accuracy	Validation_Accuracy	Train_Recall	Validation_Recall	Train_Precission	Validation_Precission	Train_F1	Validation_F1
0	CNN	1.0	0.978947	1.0	0.978947	1.0	0.979807	1.0	0.978943

show_performance_metrics(history_cnn_gray, 'CNN:Grapy', model_1_gray, X_train_normalized_gray, y_train_gray, X_val_normalized_gray, y_val_gray)

14/14 ━━━━━━━━━━━━━━━━━━━━ 0s 12ms/step
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 19ms/step
14/14 ━━━━━━━━━━━━━━━━━━━━ 0s 12ms/step
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 20ms/step

Classification Report
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 18ms/step
              precision    recall  f1-score   support

 No Helmet 0       1.00      0.98      0.99        48
    Helmet 1       0.98      1.00      0.99        47

    accuracy                           0.99        95
   macro avg       0.99      0.99      0.99        95
weighted avg       0.99      0.99      0.99        95

	model	Train_Accuracy	Validation_Accuracy	Train_Recall	Validation_Recall	Train_Precission	Validation_Precission	Train_F1	Validation_F1
0	CNN	1.000000	0.978947	1.000000	0.978947	1.000000	0.979807	1.000000	0.978943
1	CNN:Grapy	0.997732	0.989474	0.997732	0.989474	0.997743	0.989693	0.997732	0.989474

Vizualizing the predictions

show_visulization(X_val, X_val_normalized, y_val, model_1, IMAGE_WIDTH, IMAGE_HEIGHT, IMAGE_CHANNEL, index=[2, 10, 23])

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 39ms/step
Predicted Probability: 1.8098637156072073e-05, and  Label 0
True Label: 0

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 39ms/step
Predicted Probability: 0.9999996423721313, and  Label 1
True Label: 0

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 40ms/step
Predicted Probability: 1.0, and  Label 1
True Label: 1

show_visulization(X_val_gray, X_val_normalized_gray, y_val_gray, model_1_gray, IMAGE_WIDTH, IMAGE_HEIGHT, IMAGE_CHANNEL, index=[2, 10, 23])

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 39ms/step
Predicted Probability: 0.0020348140969872475, and  Label 0
True Label: 0

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 38ms/step
Predicted Probability: 0.9999998807907104, and  Label 1
True Label: 0

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 45ms/step
Predicted Probability: 1.0, and  Label 1
True Label: 1

Observation

• Our initial model uses 3 convolution layers 64 filters (3X3), and max pooling (2X2), with padding.
• It uses relu activation function.
• Later it uses 3 hidden neural network with binary output later.
• Output of the convolution layers have been flatten and feed to the final neural network.
• We found that CNN model with color images has very good accuracy, recall, precission, F1 against training, almost 100% and validation, almost 98%.
• We found that CNN model with gray images has very good accuracy, recall, precission, F1 against training, almost 99.7% and validation, almost 99%.
• We visualized some images (index 2, 10, 23 randomly) and predicted them. The model correctly predicted them.

Model CNN with Dropout

history_cnn_drop, model_1_1, history_cnn_drop_gray, model_1_1_gray = get_cnn_model(drop_out_rate= dropout_rate)
show_performance_metrics(history_cnn_drop, 'CNN (Drop=.5)', model_1_1, X_train_normalized, y_train, X_val_normalized, y_val)

Model: "sequential_16"

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ conv2d_24 (Conv2D)              │ (None, 64, 64, 64)     │         1,792 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ max_pooling2d_24 (MaxPooling2D) │ (None, 32, 32, 64)     │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dropout_4 (Dropout)             │ (None, 128)            │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ conv2d_25 (Conv2D)              │ (None, 32, 32, 64)     │        36,928 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ max_pooling2d_25 (MaxPooling2D) │ (None, 16, 16, 64)     │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dropout_4 (Dropout)             │ (None, 128)            │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ conv2d_26 (Conv2D)              │ (None, 16, 16, 64)     │        36,928 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ max_pooling2d_26 (MaxPooling2D) │ (None, 8, 8, 64)       │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ flatten_16 (Flatten)            │ (None, 4096)           │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_52 (Dense)                │ (None, 256)            │     1,048,832 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dropout_4 (Dropout)             │ (None, 128)            │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_53 (Dense)                │ (None, 128)            │        32,896 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dropout_4 (Dropout)             │ (None, 128)            │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_54 (Dense)                │ (None, 64)             │         8,256 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_55 (Dense)                │ (None, 1)              │            65 │
└─────────────────────────────────┴────────────────────────┴───────────────┘

 Total params: 1,165,697 (4.45 MB)

 Trainable params: 1,165,697 (4.45 MB)

 Non-trainable params: 0 (0.00 B)

Epoch 1/10
7/7 - 1s - 146ms/step - accuracy: 0.5556 - loss: 0.7328 - recall: 0.5069 - val_accuracy: 0.4947 - val_loss: 0.6880 - val_recall: 1.0000
Epoch 2/10
7/7 - 1s - 133ms/step - accuracy: 0.5329 - loss: 0.6972 - recall: 0.7189 - val_accuracy: 0.8526 - val_loss: 0.6869 - val_recall: 0.9787
Epoch 3/10
7/7 - 1s - 131ms/step - accuracy: 0.5533 - loss: 0.6685 - recall: 0.7742 - val_accuracy: 0.8842 - val_loss: 0.6728 - val_recall: 0.9574
Epoch 4/10
7/7 - 1s - 133ms/step - accuracy: 0.6395 - loss: 0.5962 - recall: 0.9816 - val_accuracy: 0.9053 - val_loss: 0.5851 - val_recall: 0.9574
Epoch 5/10
7/7 - 1s - 133ms/step - accuracy: 0.8617 - loss: 0.4192 - recall: 0.9447 - val_accuracy: 0.9474 - val_loss: 0.2986 - val_recall: 0.9574
Epoch 6/10
7/7 - 1s - 131ms/step - accuracy: 0.9365 - loss: 0.1876 - recall: 0.9263 - val_accuracy: 0.9789 - val_loss: 0.1297 - val_recall: 0.9787
Epoch 7/10
7/7 - 1s - 133ms/step - accuracy: 0.9410 - loss: 0.1706 - recall: 0.9401 - val_accuracy: 0.9579 - val_loss: 0.1187 - val_recall: 0.9362
Epoch 8/10
7/7 - 1s - 134ms/step - accuracy: 0.9705 - loss: 0.0962 - recall: 0.9816 - val_accuracy: 0.9368 - val_loss: 0.1551 - val_recall: 0.8936
Epoch 9/10
7/7 - 1s - 136ms/step - accuracy: 0.9683 - loss: 0.1086 - recall: 0.9631 - val_accuracy: 0.9684 - val_loss: 0.1242 - val_recall: 0.9787
Epoch 10/10
7/7 - 1s - 139ms/step - accuracy: 0.9887 - loss: 0.0609 - recall: 0.9816 - val_accuracy: 0.9789 - val_loss: 0.1145 - val_recall: 0.9787

Model: "sequential_17"

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ conv2d_27 (Conv2D)              │ (None, 64, 64, 64)     │         1,792 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ max_pooling2d_27 (MaxPooling2D) │ (None, 32, 32, 64)     │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dropout_5 (Dropout)             │ (None, 128)            │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ conv2d_28 (Conv2D)              │ (None, 32, 32, 64)     │        36,928 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ max_pooling2d_28 (MaxPooling2D) │ (None, 16, 16, 64)     │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dropout_5 (Dropout)             │ (None, 128)            │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ conv2d_29 (Conv2D)              │ (None, 16, 16, 64)     │        36,928 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ max_pooling2d_29 (MaxPooling2D) │ (None, 8, 8, 64)       │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ flatten_17 (Flatten)            │ (None, 4096)           │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_56 (Dense)                │ (None, 256)            │     1,048,832 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dropout_5 (Dropout)             │ (None, 128)            │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_57 (Dense)                │ (None, 128)            │        32,896 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dropout_5 (Dropout)             │ (None, 128)            │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_58 (Dense)                │ (None, 64)             │         8,256 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_59 (Dense)                │ (None, 1)              │            65 │
└─────────────────────────────────┴────────────────────────┴───────────────┘

 Total params: 1,165,697 (4.45 MB)

 Trainable params: 1,165,697 (4.45 MB)

 Non-trainable params: 0 (0.00 B)

Epoch 1/10
7/7 - 1s - 150ms/step - accuracy: 0.4830 - loss: 0.7484 - recall: 0.5115 - val_accuracy: 0.7684 - val_loss: 0.6884 - val_recall: 0.8511
Epoch 2/10
7/7 - 1s - 134ms/step - accuracy: 0.5261 - loss: 0.6825 - recall: 0.6544 - val_accuracy: 0.5158 - val_loss: 0.6886 - val_recall: 0.0213
Epoch 3/10
7/7 - 1s - 135ms/step - accuracy: 0.5805 - loss: 0.6688 - recall: 0.6728 - val_accuracy: 0.8526 - val_loss: 0.6811 - val_recall: 0.8085
Epoch 4/10
7/7 - 1s - 136ms/step - accuracy: 0.5986 - loss: 0.6533 - recall: 0.8479 - val_accuracy: 0.9053 - val_loss: 0.6618 - val_recall: 0.8298
Epoch 5/10
7/7 - 1s - 135ms/step - accuracy: 0.6780 - loss: 0.5988 - recall: 0.8710 - val_accuracy: 0.9579 - val_loss: 0.5503 - val_recall: 0.9362
Epoch 6/10
7/7 - 1s - 132ms/step - accuracy: 0.8503 - loss: 0.4117 - recall: 0.9309 - val_accuracy: 0.8947 - val_loss: 0.3504 - val_recall: 0.8085
Epoch 7/10
7/7 - 1s - 133ms/step - accuracy: 0.8753 - loss: 0.3082 - recall: 0.8295 - val_accuracy: 0.9789 - val_loss: 0.1997 - val_recall: 0.9787
Epoch 8/10
7/7 - 1s - 133ms/step - accuracy: 0.9297 - loss: 0.1967 - recall: 0.9401 - val_accuracy: 0.9579 - val_loss: 0.1988 - val_recall: 0.9362
Epoch 9/10
7/7 - 1s - 135ms/step - accuracy: 0.9569 - loss: 0.1480 - recall: 0.9631 - val_accuracy: 0.9474 - val_loss: 0.1876 - val_recall: 0.9149
Epoch 10/10
7/7 - 1s - 137ms/step - accuracy: 0.9592 - loss: 0.1463 - recall: 0.9309 - val_accuracy: 0.9895 - val_loss: 0.1633 - val_recall: 1.0000

14/14 ━━━━━━━━━━━━━━━━━━━━ 0s 13ms/step
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 19ms/step
14/14 ━━━━━━━━━━━━━━━━━━━━ 0s 13ms/step
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 18ms/step

Classification Report
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 18ms/step
              precision    recall  f1-score   support

 No Helmet 0       0.98      0.98      0.98        48
    Helmet 1       0.98      0.98      0.98        47

    accuracy                           0.98        95
   macro avg       0.98      0.98      0.98        95
weighted avg       0.98      0.98      0.98        95

	model	Train_Accuracy	Validation_Accuracy	Train_Recall	Validation_Recall	Train_Precission	Validation_Precission	Train_F1	Validation_F1
0	CNN	1.000000	0.978947	1.000000	0.978947	1.000000	0.979807	1.000000	0.978943
1	CNN:Grapy	0.997732	0.989474	0.997732	0.989474	0.997743	0.989693	0.997732	0.989474
2	CNN (Drop=.5)	0.993197	0.978947	0.993197	0.978947	0.993287	0.978947	0.993196	0.978947

show_performance_metrics(history_cnn_drop_gray, 'CNN (Drop=.5): Gray', model_1_1_gray, X_train_normalized_gray, y_train_gray, X_val_normalized_gray, y_val_gray)

14/14 ━━━━━━━━━━━━━━━━━━━━ 0s 13ms/step
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 19ms/step
14/14 ━━━━━━━━━━━━━━━━━━━━ 0s 14ms/step
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 20ms/step

Classification Report
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 21ms/step
              precision    recall  f1-score   support

 No Helmet 0       1.00      0.98      0.99        48
    Helmet 1       0.98      1.00      0.99        47

    accuracy                           0.99        95
   macro avg       0.99      0.99      0.99        95
weighted avg       0.99      0.99      0.99        95

	model	Train_Accuracy	Validation_Accuracy	Train_Recall	Validation_Recall	Train_Precission	Validation_Precission	Train_F1	Validation_F1
0	CNN	1.000000	0.978947	1.000000	0.978947	1.000000	0.979807	1.000000	0.978943
1	CNN:Grapy	0.997732	0.989474	0.997732	0.989474	0.997743	0.989693	0.997732	0.989474
2	CNN (Drop=.5)	0.993197	0.978947	0.993197	0.978947	0.993287	0.978947	0.993196	0.978947
3	CNN (Drop=.5): Gray	0.979592	0.989474	0.979592	0.989474	0.979603	0.989693	0.979592	0.989474

Vizualizing the predictions

show_visulization(X_val, X_val_normalized, y_val, model_1_1, IMAGE_WIDTH, IMAGE_HEIGHT, IMAGE_CHANNEL, index=[2, 10, 23])

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 41ms/step
Predicted Probability: 0.01702054589986801, and  Label 0
True Label: 0

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 39ms/step
Predicted Probability: 0.9909351468086243, and  Label 1
True Label: 0

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 41ms/step
Predicted Probability: 0.999834418296814, and  Label 1
True Label: 1

show_visulization(X_val_gray, X_val_normalized_gray, y_val_gray, model_1_1_gray, IMAGE_WIDTH, IMAGE_HEIGHT, IMAGE_CHANNEL, index=[2, 10, 23])

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 39ms/step
Predicted Probability: 0.08309952169656754, and  Label 0
True Label: 0

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 38ms/step
Predicted Probability: 0.9992927312850952, and  Label 1
True Label: 0

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 38ms/step
Predicted Probability: 0.9999877214431763, and  Label 1
True Label: 1

Observation

• Our initial model uses 3 convolution layers 64 filters (3X3), and max pooling (2X2), with padding.
• It uses relu activation function.
• Later it uses 3 hidden neural network with binary output later.
• Output of the convolution layers have been flatten and feed to the final neural network.
• We added 50% drop out in both CNN and neural network
• We found that CNN color image model with drop out of 50% has very good accuracy and recall against training, almost 99% and validation, almost 97.8%.
• We found that CNN gray image model with drop out of 50% has very good accuracy and recall against training, almost 98% and validation, almost 99%.
• We visualized some images (index 2, 10, 23 randomly) and predicted them. The model correctly predicted them.

Model 2: (VGG-16 (Base))

• We will be loading a pre-built architecture - VGG16, which was trained on the ImageNet dataset and is the runner-up in the ImageNet competition in 2014.

• For training VGG16, we will directly use the convolutional and pooling layers and freeze their weights i.e. no training will be done on them. For classification, we will add a Flatten and a single dense layer.

vgg_model = VGG16(weights='imagenet',include_top=False,input_shape=(IMAGE_WIDTH, IMAGE_HEIGHT,IMAGE_CHANNEL))
vgg_model.summary()

Model: "vgg16"

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ input_layer_19 (InputLayer)     │ (None, 64, 64, 3)      │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block1_conv1 (Conv2D)           │ (None, 64, 64, 64)     │         1,792 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block1_conv2 (Conv2D)           │ (None, 64, 64, 64)     │        36,928 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block1_pool (MaxPooling2D)      │ (None, 32, 32, 64)     │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block2_conv1 (Conv2D)           │ (None, 32, 32, 128)    │        73,856 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block2_conv2 (Conv2D)           │ (None, 32, 32, 128)    │       147,584 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block2_pool (MaxPooling2D)      │ (None, 16, 16, 128)    │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block3_conv1 (Conv2D)           │ (None, 16, 16, 256)    │       295,168 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block3_conv2 (Conv2D)           │ (None, 16, 16, 256)    │       590,080 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block3_conv3 (Conv2D)           │ (None, 16, 16, 256)    │       590,080 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block3_pool (MaxPooling2D)      │ (None, 8, 8, 256)      │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block4_conv1 (Conv2D)           │ (None, 8, 8, 512)      │     1,180,160 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block4_conv2 (Conv2D)           │ (None, 8, 8, 512)      │     2,359,808 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block4_conv3 (Conv2D)           │ (None, 8, 8, 512)      │     2,359,808 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block4_pool (MaxPooling2D)      │ (None, 4, 4, 512)      │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block5_conv1 (Conv2D)           │ (None, 4, 4, 512)      │     2,359,808 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block5_conv2 (Conv2D)           │ (None, 4, 4, 512)      │     2,359,808 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block5_conv3 (Conv2D)           │ (None, 4, 4, 512)      │     2,359,808 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block5_pool (MaxPooling2D)      │ (None, 2, 2, 512)      │             0 │
└─────────────────────────────────┴────────────────────────┴───────────────┘

 Total params: 14,714,688 (56.13 MB)

 Trainable params: 14,714,688 (56.13 MB)

 Non-trainable params: 0 (0.00 B)

def get_vgg_color_model(only_base=True, train_datagen = ImageDataGenerator(), drop_out_rate = None):
  # Making all the layers of the VGG model non-trainable. i.e. freezing them
  for layer in vgg_model.layers:
      layer.trainable = False

  if drop_out_rate is not None:
    drop_out = Dropout(rate=drop_out_rate)

  model_2 = Sequential()

  # Adding the convolutional part of the VGG16 model from above
  model_2.add(vgg_model)

  # Flattening the output of the VGG16 model because it is from a convolutional layer
  model_2.add(Flatten())

  if only_base == False:
    # Adding a dense output layer
    model_2.add(Dense(256, activation='relu', kernel_initializer='he_uniform'))
    if drop_out_rate is not None:
      model_2.add(drop_out)

    model_2.add(Dense(128, activation='relu', kernel_initializer='he_uniform'))
    if drop_out_rate is not None:
      model_2.add(drop_out)

  # Adding a dense output layer
  model_2.add(Dense(1, activation='sigmoid'))

  opt=Adam()
  # Compile model
  # Other metrics like precision,f1_score,recall can be used
  model_2.compile(optimizer=opt, loss='binary_crossentropy', metrics=['accuracy', 'recall'])


  # Generating the summary of the model
  model_2.summary()

  #train_datagen = ImageDataGenerator()

  # Epochs
  epochs = 10
  # Batch size
  batch_size = 64

  history_vgg16 = model_2.fit(train_datagen.flow(X_train_normalized, y_train,
                                        batch_size=batch_size,
                                        seed=42,
                                        shuffle=True),
                      epochs=epochs,
                      steps_per_epoch=X_train_normalized.shape[0] // batch_size,
                      validation_data=(X_val_normalized, y_val),
                      verbose=1)

  return history_vgg16, model_2


def get_vgg_gray_model(only_base=True, train_datagen = ImageDataGenerator(), drop_out_rate = None):
  # Making all the layers of the VGG model non-trainable. i.e. freezing them
  for layer in vgg_model.layers:
      layer.trainable = False

  if drop_out_rate is not None:
    drop_out = Dropout(rate=drop_out_rate)

  model_2 = Sequential()

  # Adding the convolutional part of the VGG16 model from above
  model_2.add(vgg_model)

  # Flattening the output of the VGG16 model because it is from a convolutional layer
  model_2.add(Flatten())

  if only_base == False:
    # Adding a dense output layer
    model_2.add(Dense(256, activation='relu', kernel_initializer='he_uniform'))
    if drop_out_rate is not None:
      model_2.add(drop_out)

    model_2.add(Dense(128, activation='relu', kernel_initializer='he_uniform'))
    if drop_out_rate is not None:
      model_2.add(drop_out)

  # Adding a dense output layer
  model_2.add(Dense(1, activation='sigmoid'))

  opt=Adam()
  # Compile model
  # Other metrics like precision,f1_score,recall can be used
  model_2.compile(optimizer=opt, loss='binary_crossentropy', metrics=['accuracy', 'recall'])


  # Generating the summary of the model
  model_2.summary()

  #train_datagen = ImageDataGenerator()

  # Epochs
  epochs = 10
  # Batch size
  batch_size = 64

  tf.config.run_functions_eagerly(True)

  history_vgg16 = model_2.fit(train_datagen.flow(X_train_normalized_gray, y_train_gray,
                                        batch_size=batch_size,
                                        seed=42,
                                        shuffle=True),
                      epochs=epochs,
                      steps_per_epoch=X_train_normalized_gray.shape[0] // batch_size,
                      validation_data=(X_val_normalized_gray, y_val_gray),
                      verbose=1)




  return history_vgg16, model_2

def get_vgg_model(only_base=True, train_datagen = ImageDataGenerator(), drop_out_rate = None):
  history_vgg16, model_2 = get_vgg_color_model(only_base, train_datagen, drop_out_rate)
  history_vgg16_gray, model_2_gray = get_vgg_gray_model(only_base, train_datagen, drop_out_rate)
  return history_vgg16, model_2, history_vgg16_gray, model_2_gray

# Getting the base vgg 16 model
history_vgg16_base, model_vgg_base, history_vgg16_base_gray, model_vgg_base_gray = get_vgg_model(only_base=True)

Model: "sequential_18"

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ vgg16 (Functional)              │ (None, 2, 2, 512)      │    14,714,688 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ flatten_18 (Flatten)            │ (None, 2048)           │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_60 (Dense)                │ (None, 1)              │         2,049 │
└─────────────────────────────────┴────────────────────────┴───────────────┘

 Total params: 14,716,737 (56.14 MB)

 Trainable params: 2,049 (8.00 KB)

 Non-trainable params: 14,714,688 (56.13 MB)

Epoch 1/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 152ms/step - accuracy: 0.5551 - loss: 0.7327 - recall: 1.0000 - val_accuracy: 0.9053 - val_loss: 0.4571 - val_recall: 0.8511
Epoch 2/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 32ms/step - accuracy: 0.9375 - loss: 0.4376 - recall: 0.9706 - val_accuracy: 0.9053 - val_loss: 0.4351 - val_recall: 0.8298
Epoch 3/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 145ms/step - accuracy: 0.9416 - loss: 0.3885 - recall: 0.8864 - val_accuracy: 0.9158 - val_loss: 0.3219 - val_recall: 0.8298
Epoch 4/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 33ms/step - accuracy: 0.9474 - loss: 0.2518 - recall: 0.8500 - val_accuracy: 0.9368 - val_loss: 0.3011 - val_recall: 0.8723
Epoch 5/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 143ms/step - accuracy: 0.9810 - loss: 0.2524 - recall: 0.9641 - val_accuracy: 0.9789 - val_loss: 0.2009 - val_recall: 0.9787
Epoch 6/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 32ms/step - accuracy: 0.9844 - loss: 0.1873 - recall: 0.9688 - val_accuracy: 0.9895 - val_loss: 0.1904 - val_recall: 1.0000
Epoch 7/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 144ms/step - accuracy: 0.9967 - loss: 0.1638 - recall: 0.9949 - val_accuracy: 0.9789 - val_loss: 0.1510 - val_recall: 1.0000
Epoch 8/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 32ms/step - accuracy: 0.9844 - loss: 0.1584 - recall: 1.0000 - val_accuracy: 0.9789 - val_loss: 0.1464 - val_recall: 1.0000
Epoch 9/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 143ms/step - accuracy: 0.9833 - loss: 0.1330 - recall: 1.0000 - val_accuracy: 0.9789 - val_loss: 0.1219 - val_recall: 1.0000
Epoch 10/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 32ms/step - accuracy: 1.0000 - loss: 0.0929 - recall: 1.0000 - val_accuracy: 0.9789 - val_loss: 0.1186 - val_recall: 1.0000

Model: "sequential_19"

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ vgg16 (Functional)              │ (None, 2, 2, 512)      │    14,714,688 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ flatten_19 (Flatten)            │ (None, 2048)           │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_61 (Dense)                │ (None, 1)              │         2,049 │
└─────────────────────────────────┴────────────────────────┴───────────────┘

 Total params: 14,716,737 (56.14 MB)

 Trainable params: 2,049 (8.00 KB)

 Non-trainable params: 14,714,688 (56.13 MB)

Epoch 1/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 151ms/step - accuracy: 0.6164 - loss: 0.6766 - recall: 0.8440 - val_accuracy: 0.9053 - val_loss: 0.5090 - val_recall: 0.8511
Epoch 2/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 32ms/step - accuracy: 0.9062 - loss: 0.4821 - recall: 0.8571 - val_accuracy: 0.9368 - val_loss: 0.4754 - val_recall: 0.9149
Epoch 3/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 144ms/step - accuracy: 0.9188 - loss: 0.4283 - recall: 0.8741 - val_accuracy: 0.9684 - val_loss: 0.3214 - val_recall: 0.9574
Epoch 4/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 33ms/step - accuracy: 1.0000 - loss: 0.3081 - recall: 1.0000 - val_accuracy: 0.9789 - val_loss: 0.3023 - val_recall: 0.9787
Epoch 5/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 143ms/step - accuracy: 0.9945 - loss: 0.2532 - recall: 0.9924 - val_accuracy: 0.9789 - val_loss: 0.2167 - val_recall: 0.9787
Epoch 6/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 35ms/step - accuracy: 0.9844 - loss: 0.2084 - recall: 0.9706 - val_accuracy: 0.9684 - val_loss: 0.2063 - val_recall: 0.9574
Epoch 7/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 146ms/step - accuracy: 0.9921 - loss: 0.1848 - recall: 0.9831 - val_accuracy: 0.9684 - val_loss: 0.1589 - val_recall: 0.9574
Epoch 8/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 33ms/step - accuracy: 0.9844 - loss: 0.1504 - recall: 0.9655 - val_accuracy: 0.9684 - val_loss: 0.1530 - val_recall: 0.9574
Epoch 9/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 148ms/step - accuracy: 0.9917 - loss: 0.1346 - recall: 0.9837 - val_accuracy: 0.9895 - val_loss: 0.1256 - val_recall: 1.0000
Epoch 10/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 32ms/step - accuracy: 0.9825 - loss: 0.1250 - recall: 0.9655 - val_accuracy: 0.9895 - val_loss: 0.1221 - val_recall: 1.0000

show_performance_metrics(history_vgg16_base, 'VGG_Base', model_vgg_base, X_train_normalized, y_train, X_val_normalized, y_val)

14/14 ━━━━━━━━━━━━━━━━━━━━ 0s 24ms/step
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 30ms/step
14/14 ━━━━━━━━━━━━━━━━━━━━ 0s 24ms/step
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 31ms/step

Classification Report
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 30ms/step
              precision    recall  f1-score   support

 No Helmet 0       1.00      0.96      0.98        48
    Helmet 1       0.96      1.00      0.98        47

    accuracy                           0.98        95
   macro avg       0.98      0.98      0.98        95
weighted avg       0.98      0.98      0.98        95

	model	Train_Accuracy	Validation_Accuracy	Train_Recall	Validation_Recall	Train_Precission	Validation_Precission	Train_F1	Validation_F1
0	CNN	1.000000	0.978947	1.000000	0.978947	1.000000	0.979807	1.000000	0.978943
1	CNN:Grapy	0.997732	0.989474	0.997732	0.989474	0.997743	0.989693	0.997732	0.989474
2	CNN (Drop=.5)	0.993197	0.978947	0.993197	0.978947	0.993287	0.978947	0.993196	0.978947
3	CNN (Drop=.5): Gray	0.979592	0.989474	0.979592	0.989474	0.979603	0.989693	0.979592	0.989474
4	VGG_Base	0.995465	0.978947	0.995465	0.978947	0.995506	0.979807	0.995465	0.978943

show_performance_metrics(history_vgg16_base_gray, 'VGG_Base: Gray', model_vgg_base_gray, X_train_normalized_gray, y_train_gray, X_val_normalized_gray, y_val_gray)

14/14 ━━━━━━━━━━━━━━━━━━━━ 0s 24ms/step
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 31ms/step
14/14 ━━━━━━━━━━━━━━━━━━━━ 0s 24ms/step
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 31ms/step

Classification Report
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 31ms/step
              precision    recall  f1-score   support

 No Helmet 0       1.00      0.98      0.99        48
    Helmet 1       0.98      1.00      0.99        47

    accuracy                           0.99        95
   macro avg       0.99      0.99      0.99        95
weighted avg       0.99      0.99      0.99        95

	model	Train_Accuracy	Validation_Accuracy	Train_Recall	Validation_Recall	Train_Precission	Validation_Precission	Train_F1	Validation_F1
0	CNN	1.000000	0.978947	1.000000	0.978947	1.000000	0.979807	1.000000	0.978943
1	CNN:Grapy	0.997732	0.989474	0.997732	0.989474	0.997743	0.989693	0.997732	0.989474
2	CNN (Drop=.5)	0.993197	0.978947	0.993197	0.978947	0.993287	0.978947	0.993196	0.978947
3	CNN (Drop=.5): Gray	0.979592	0.989474	0.979592	0.989474	0.979603	0.989693	0.979592	0.989474
4	VGG_Base	0.995465	0.978947	0.995465	0.978947	0.995506	0.979807	0.995465	0.978943
5	VGG_Base: Gray	0.995465	0.989474	0.995465	0.989474	0.995505	0.989693	0.995464	0.989474

Visualizing the prediction:

show_visulization(X_val, X_val_normalized, y_val, model_vgg_base, IMAGE_WIDTH, IMAGE_HEIGHT, IMAGE_CHANNEL, index=[2, 10, 23])

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 56ms/step
Predicted Probability: 0.1277976781129837, and  Label 0
True Label: 0

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 54ms/step
Predicted Probability: 0.576281726360321, and  Label 1
True Label: 0

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 58ms/step
Predicted Probability: 0.972450852394104, and  Label 1
True Label: 1

show_visulization(X_val_gray, X_val_normalized_gray, y_val_gray, model_vgg_base_gray, IMAGE_WIDTH, IMAGE_HEIGHT, IMAGE_CHANNEL, index=[2, 10, 23])

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 54ms/step
Predicted Probability: 0.1768014132976532, and  Label 0
True Label: 0

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 56ms/step
Predicted Probability: 0.6251361966133118, and  Label 1
True Label: 0

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 54ms/step
Predicted Probability: 0.9549097418785095, and  Label 1
True Label: 1

Observation

• We freeze VGG 16 convolution layers and feed it to an output layer.

• We found that VGG base color image model has very good accuracy and recall against training (99.5%) and validation, almost 97.8%.

• We found that VGG base gray image model has very good accuracy and recall against training (99.5%) and validation, almost 99%.

• We visualized some images (index 2, 10, 23 randomly) and predicted them. The model correctly predicted them.

Model 3: (VGG-16 (Base + FFNN))

# Getting the base vgg 16 model with FFNN
history_vgg16_ffnn, model_vgg_ffnn, history_vgg16_ffnn_gray, model_vgg_ffnn_gray = get_vgg_model(only_base=False)

Model: "sequential_20"

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ vgg16 (Functional)              │ (None, 2, 2, 512)      │    14,714,688 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ flatten_20 (Flatten)            │ (None, 2048)           │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_62 (Dense)                │ (None, 256)            │       524,544 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_63 (Dense)                │ (None, 128)            │        32,896 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_64 (Dense)                │ (None, 1)              │           129 │
└─────────────────────────────────┴────────────────────────┴───────────────┘

 Total params: 15,272,257 (58.26 MB)

 Trainable params: 557,569 (2.13 MB)

 Non-trainable params: 14,714,688 (56.13 MB)

Epoch 1/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 158ms/step - accuracy: 0.8051 - loss: 0.3897 - recall: 0.9520 - val_accuracy: 0.9789 - val_loss: 0.0412 - val_recall: 0.9574
Epoch 2/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 36ms/step - accuracy: 0.9844 - loss: 0.0300 - recall: 0.9667 - val_accuracy: 0.9895 - val_loss: 0.0309 - val_recall: 1.0000
Epoch 3/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 149ms/step - accuracy: 0.9964 - loss: 0.0114 - recall: 0.9927 - val_accuracy: 0.9789 - val_loss: 0.0490 - val_recall: 1.0000
Epoch 4/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 35ms/step - accuracy: 1.0000 - loss: 0.0025 - recall: 1.0000 - val_accuracy: 0.9789 - val_loss: 0.0450 - val_recall: 1.0000
Epoch 5/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 147ms/step - accuracy: 1.0000 - loss: 0.0029 - recall: 1.0000 - val_accuracy: 0.9895 - val_loss: 0.0127 - val_recall: 1.0000
Epoch 6/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 36ms/step - accuracy: 1.0000 - loss: 1.9395e-04 - recall: 1.0000 - val_accuracy: 1.0000 - val_loss: 0.0114 - val_recall: 1.0000
Epoch 7/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 147ms/step - accuracy: 1.0000 - loss: 6.7723e-04 - recall: 1.0000 - val_accuracy: 1.0000 - val_loss: 0.0101 - val_recall: 1.0000
Epoch 8/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 38ms/step - accuracy: 1.0000 - loss: 2.8148e-04 - recall: 1.0000 - val_accuracy: 1.0000 - val_loss: 0.0101 - val_recall: 1.0000
Epoch 9/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 150ms/step - accuracy: 1.0000 - loss: 1.7061e-04 - recall: 1.0000 - val_accuracy: 0.9895 - val_loss: 0.0110 - val_recall: 1.0000
Epoch 10/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 38ms/step - accuracy: 1.0000 - loss: 3.7717e-05 - recall: 1.0000 - val_accuracy: 0.9895 - val_loss: 0.0115 - val_recall: 1.0000

Model: "sequential_21"

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ vgg16 (Functional)              │ (None, 2, 2, 512)      │    14,714,688 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ flatten_21 (Flatten)            │ (None, 2048)           │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_65 (Dense)                │ (None, 256)            │       524,544 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_66 (Dense)                │ (None, 128)            │        32,896 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_67 (Dense)                │ (None, 1)              │           129 │
└─────────────────────────────────┴────────────────────────┴───────────────┘

 Total params: 15,272,257 (58.26 MB)

 Trainable params: 557,569 (2.13 MB)

 Non-trainable params: 14,714,688 (56.13 MB)

Epoch 1/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 159ms/step - accuracy: 0.6641 - loss: 0.4582 - recall: 0.6952 - val_accuracy: 0.9789 - val_loss: 0.0609 - val_recall: 0.9574
Epoch 2/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 36ms/step - accuracy: 0.9688 - loss: 0.0799 - recall: 0.9429 - val_accuracy: 0.9789 - val_loss: 0.0415 - val_recall: 1.0000
Epoch 3/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 148ms/step - accuracy: 0.9928 - loss: 0.0243 - recall: 1.0000 - val_accuracy: 0.9789 - val_loss: 0.0313 - val_recall: 1.0000
Epoch 4/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 34ms/step - accuracy: 1.0000 - loss: 0.0035 - recall: 1.0000 - val_accuracy: 0.9895 - val_loss: 0.0261 - val_recall: 1.0000
Epoch 5/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 147ms/step - accuracy: 0.9988 - loss: 0.0055 - recall: 0.9975 - val_accuracy: 0.9895 - val_loss: 0.0192 - val_recall: 1.0000
Epoch 6/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 36ms/step - accuracy: 1.0000 - loss: 0.0013 - recall: 1.0000 - val_accuracy: 0.9895 - val_loss: 0.0200 - val_recall: 1.0000
Epoch 7/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 148ms/step - accuracy: 1.0000 - loss: 9.8625e-04 - recall: 1.0000 - val_accuracy: 0.9895 - val_loss: 0.0248 - val_recall: 1.0000
Epoch 8/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 34ms/step - accuracy: 1.0000 - loss: 0.0034 - recall: 1.0000 - val_accuracy: 0.9895 - val_loss: 0.0257 - val_recall: 1.0000
Epoch 9/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 148ms/step - accuracy: 1.0000 - loss: 5.8375e-04 - recall: 1.0000 - val_accuracy: 0.9895 - val_loss: 0.0257 - val_recall: 1.0000
Epoch 10/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 35ms/step - accuracy: 1.0000 - loss: 3.9772e-04 - recall: 1.0000 - val_accuracy: 0.9895 - val_loss: 0.0250 - val_recall: 1.0000

show_performance_metrics(history_vgg16_ffnn, 'VGG_FFNN', model_vgg_ffnn, X_train_normalized, y_train, X_val_normalized, y_val)

14/14 ━━━━━━━━━━━━━━━━━━━━ 0s 26ms/step
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 33ms/step
14/14 ━━━━━━━━━━━━━━━━━━━━ 0s 27ms/step
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 35ms/step

Classification Report
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 36ms/step
              precision    recall  f1-score   support

 No Helmet 0       1.00      0.98      0.99        48
    Helmet 1       0.98      1.00      0.99        47

    accuracy                           0.99        95
   macro avg       0.99      0.99      0.99        95
weighted avg       0.99      0.99      0.99        95

	model	Train_Accuracy	Validation_Accuracy	Train_Recall	Validation_Recall	Train_Precission	Validation_Precission	Train_F1	Validation_F1
0	CNN	1.000000	0.978947	1.000000	0.978947	1.000000	0.979807	1.000000	0.978943
1	CNN:Grapy	0.997732	0.989474	0.997732	0.989474	0.997743	0.989693	0.997732	0.989474
2	CNN (Drop=.5)	0.993197	0.978947	0.993197	0.978947	0.993287	0.978947	0.993196	0.978947
3	CNN (Drop=.5): Gray	0.979592	0.989474	0.979592	0.989474	0.979603	0.989693	0.979592	0.989474
4	VGG_Base	0.995465	0.978947	0.995465	0.978947	0.995506	0.979807	0.995465	0.978943
5	VGG_Base: Gray	0.995465	0.989474	0.995465	0.989474	0.995505	0.989693	0.995464	0.989474
6	VGG_FFNN	1.000000	0.989474	1.000000	0.989474	1.000000	0.989693	1.000000	0.989474

show_performance_metrics(history_vgg16_ffnn_gray, 'VGG_FFNN: GRAY', model_vgg_ffnn_gray, X_train_normalized_gray, y_train_gray, X_val_normalized_gray, y_val_gray)

14/14 ━━━━━━━━━━━━━━━━━━━━ 0s 26ms/step
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 32ms/step
14/14 ━━━━━━━━━━━━━━━━━━━━ 0s 26ms/step
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 32ms/step

Classification Report
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 32ms/step
              precision    recall  f1-score   support

 No Helmet 0       1.00      0.98      0.99        48
    Helmet 1       0.98      1.00      0.99        47

    accuracy                           0.99        95
   macro avg       0.99      0.99      0.99        95
weighted avg       0.99      0.99      0.99        95

	model	Train_Accuracy	Validation_Accuracy	Train_Recall	Validation_Recall	Train_Precission	Validation_Precission	Train_F1	Validation_F1
0	CNN	1.000000	0.978947	1.000000	0.978947	1.000000	0.979807	1.000000	0.978943
1	CNN:Grapy	0.997732	0.989474	0.997732	0.989474	0.997743	0.989693	0.997732	0.989474
2	CNN (Drop=.5)	0.993197	0.978947	0.993197	0.978947	0.993287	0.978947	0.993196	0.978947
3	CNN (Drop=.5): Gray	0.979592	0.989474	0.979592	0.989474	0.979603	0.989693	0.979592	0.989474
4	VGG_Base	0.995465	0.978947	0.995465	0.978947	0.995506	0.979807	0.995465	0.978943
5	VGG_Base: Gray	0.995465	0.989474	0.995465	0.989474	0.995505	0.989693	0.995464	0.989474
6	VGG_FFNN	1.000000	0.989474	1.000000	0.989474	1.000000	0.989693	1.000000	0.989474
7	VGG_FFNN: GRAY	1.000000	0.989474	1.000000	0.989474	1.000000	0.989693	1.000000	0.989474

Visualizing the predictions

show_visulization(X_val, X_val_normalized, y_val, model_vgg_ffnn, IMAGE_WIDTH, IMAGE_HEIGHT, IMAGE_CHANNEL, index=[2, 10, 23])

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 55ms/step
Predicted Probability: 1.7306635413660842e-07, and  Label 0
True Label: 0

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 53ms/step
Predicted Probability: 0.5535781383514404, and  Label 1
True Label: 0

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 54ms/step
Predicted Probability: 0.9999998807907104, and  Label 1
True Label: 1

show_visulization(X_val_gray, X_val_normalized_gray, y_val_gray, model_vgg_ffnn_gray, IMAGE_WIDTH, IMAGE_HEIGHT, IMAGE_CHANNEL, index=[2, 10, 23])

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 52ms/step
Predicted Probability: 3.8580077671213076e-05, and  Label 0
True Label: 0

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 53ms/step
Predicted Probability: 0.8816477656364441, and  Label 1
True Label: 0

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 52ms/step
Predicted Probability: 1.0, and  Label 1
True Label: 1

Observation

• We freeze VGG 16 convolution layers , add fully connected neural network with 2 hidden layers and feed it to an output layer.
• FNN uses relu activation function and batch normaliztion (he) and adam optimizer.
• We found that colored model has very good accuracy, recall, precission, f1 against training almost 100% and in validation, almost 99%.
• We found that gray model has very good accuracy, recall, precission, f1 against training almost 100% and in validation, almost 99%.
• We visualized some images (index 2, 10, 23 randomly) and predicted them. The model correctly predicted them.

Model 4: (VGG-16 (Base + FFNN + Data Augmentation)

• In most of the real-world case studies, it is challenging to acquire a large number of images and then train CNNs.

• To overcome this problem, one approach we might consider is Data Augmentation.

• CNNs have the property of translational invariance, which means they can recognise an object even if its appearance shifts translationally in some way. - Taking this attribute into account, we can augment the images using the techniques listed below

  • Horizontal Flip (should be set to True/False)
  • Vertical Flip (should be set to True/False)
  • Height Shift (should be between 0 and 1)
  • Width Shift (should be between 0 and 1)
  • Rotation (should be between 0 and 180)
  • Shear (should be between 0 and 1)
  • Zoom (should be between 0 and 1) etc.

Remember, data augmentation should not be used in the validation/test data set.

# Getting the base vgg 16 model with FFNN
# Applying data augmentation
train_datagen = ImageDataGenerator(
                              rotation_range=20,
                              fill_mode='nearest',
                              width_shift_range=0.2,
                              height_shift_range=0.2,
                              shear_range=0.3,
                              zoom_range=0.4
                              )

history_vgg16_aug, model_vgg_aug, history_vgg16_aug_gray, model_vgg_aug_gray = get_vgg_model(only_base=False, train_datagen=train_datagen)

Model: "sequential_22"

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ vgg16 (Functional)              │ (None, 2, 2, 512)      │    14,714,688 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ flatten_22 (Flatten)            │ (None, 2048)           │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_68 (Dense)                │ (None, 256)            │       524,544 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_69 (Dense)                │ (None, 128)            │        32,896 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_70 (Dense)                │ (None, 1)              │           129 │
└─────────────────────────────────┴────────────────────────┴───────────────┘

 Total params: 15,272,257 (58.26 MB)

 Trainable params: 557,569 (2.13 MB)

 Non-trainable params: 14,714,688 (56.13 MB)

Epoch 1/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 168ms/step - accuracy: 0.6564 - loss: 0.7287 - recall: 0.6327 - val_accuracy: 0.9684 - val_loss: 0.1207 - val_recall: 0.9574
Epoch 2/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 36ms/step - accuracy: 0.9844 - loss: 0.1383 - recall: 0.9677 - val_accuracy: 0.9684 - val_loss: 0.0881 - val_recall: 0.9787
Epoch 3/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 161ms/step - accuracy: 0.9620 - loss: 0.1024 - recall: 0.9724 - val_accuracy: 0.9789 - val_loss: 0.0504 - val_recall: 0.9787
Epoch 4/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 34ms/step - accuracy: 1.0000 - loss: 0.0107 - recall: 1.0000 - val_accuracy: 0.9789 - val_loss: 0.0491 - val_recall: 0.9787
Epoch 5/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 154ms/step - accuracy: 0.9898 - loss: 0.0333 - recall: 0.9876 - val_accuracy: 0.9789 - val_loss: 0.0476 - val_recall: 0.9787
Epoch 6/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 34ms/step - accuracy: 0.9688 - loss: 0.0360 - recall: 0.9714 - val_accuracy: 0.9789 - val_loss: 0.0458 - val_recall: 0.9787
Epoch 7/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 152ms/step - accuracy: 0.9877 - loss: 0.0294 - recall: 0.9836 - val_accuracy: 0.9895 - val_loss: 0.0537 - val_recall: 1.0000
Epoch 8/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 35ms/step - accuracy: 0.9825 - loss: 0.0279 - recall: 1.0000 - val_accuracy: 0.9895 - val_loss: 0.0511 - val_recall: 1.0000
Epoch 9/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 155ms/step - accuracy: 0.9964 - loss: 0.0126 - recall: 0.9928 - val_accuracy: 0.9789 - val_loss: 0.0418 - val_recall: 0.9787
Epoch 10/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 36ms/step - accuracy: 1.0000 - loss: 0.0144 - recall: 1.0000 - val_accuracy: 0.9789 - val_loss: 0.0421 - val_recall: 0.9787

Model: "sequential_23"

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ vgg16 (Functional)              │ (None, 2, 2, 512)      │    14,714,688 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ flatten_23 (Flatten)            │ (None, 2048)           │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_71 (Dense)                │ (None, 256)            │       524,544 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_72 (Dense)                │ (None, 128)            │        32,896 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_73 (Dense)                │ (None, 1)              │           129 │
└─────────────────────────────────┴────────────────────────┴───────────────┘

 Total params: 15,272,257 (58.26 MB)

 Trainable params: 557,569 (2.13 MB)

 Non-trainable params: 14,714,688 (56.13 MB)

Epoch 1/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 162ms/step - accuracy: 0.7155 - loss: 0.5536 - recall: 0.6189 - val_accuracy: 0.9789 - val_loss: 0.0704 - val_recall: 1.0000
Epoch 2/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 35ms/step - accuracy: 1.0000 - loss: 0.0773 - recall: 1.0000 - val_accuracy: 0.9789 - val_loss: 0.0588 - val_recall: 1.0000
Epoch 3/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 155ms/step - accuracy: 0.9584 - loss: 0.1000 - recall: 0.9726 - val_accuracy: 0.9895 - val_loss: 0.0371 - val_recall: 1.0000
Epoch 4/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 34ms/step - accuracy: 0.9825 - loss: 0.0341 - recall: 0.9643 - val_accuracy: 0.9895 - val_loss: 0.0365 - val_recall: 1.0000
Epoch 5/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 155ms/step - accuracy: 0.9791 - loss: 0.0768 - recall: 0.9884 - val_accuracy: 0.9895 - val_loss: 0.0378 - val_recall: 1.0000
Epoch 6/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 34ms/step - accuracy: 0.9844 - loss: 0.0483 - recall: 1.0000 - val_accuracy: 0.9895 - val_loss: 0.0406 - val_recall: 1.0000
Epoch 7/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 153ms/step - accuracy: 0.9948 - loss: 0.0201 - recall: 1.0000 - val_accuracy: 0.9895 - val_loss: 0.0487 - val_recall: 1.0000
Epoch 8/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 34ms/step - accuracy: 0.9844 - loss: 0.0658 - recall: 1.0000 - val_accuracy: 0.9895 - val_loss: 0.0405 - val_recall: 1.0000
Epoch 9/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 169ms/step - accuracy: 0.9817 - loss: 0.0525 - recall: 0.9686 - val_accuracy: 0.9895 - val_loss: 0.0678 - val_recall: 1.0000
Epoch 10/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 35ms/step - accuracy: 0.9531 - loss: 0.0954 - recall: 0.9706 - val_accuracy: 0.9895 - val_loss: 0.0720 - val_recall: 1.0000

show_performance_metrics(history_vgg16_aug, 'VGG_AUG', model_vgg_aug, X_train_normalized, y_train, X_val_normalized, y_val)

14/14 ━━━━━━━━━━━━━━━━━━━━ 0s 27ms/step
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 31ms/step
14/14 ━━━━━━━━━━━━━━━━━━━━ 0s 26ms/step
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 31ms/step

Classification Report
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 32ms/step
              precision    recall  f1-score   support

 No Helmet 0       0.98      0.98      0.98        48
    Helmet 1       0.98      0.98      0.98        47

    accuracy                           0.98        95
   macro avg       0.98      0.98      0.98        95
weighted avg       0.98      0.98      0.98        95

	model	Train_Accuracy	Validation_Accuracy	Train_Recall	Validation_Recall	Train_Precission	Validation_Precission	Train_F1	Validation_F1
0	CNN	1.000000	0.978947	1.000000	0.978947	1.000000	0.979807	1.000000	0.978943
1	CNN:Grapy	0.997732	0.989474	0.997732	0.989474	0.997743	0.989693	0.997732	0.989474
2	CNN (Drop=.5)	0.993197	0.978947	0.993197	0.978947	0.993287	0.978947	0.993196	0.978947
3	CNN (Drop=.5): Gray	0.979592	0.989474	0.979592	0.989474	0.979603	0.989693	0.979592	0.989474
4	VGG_Base	0.995465	0.978947	0.995465	0.978947	0.995506	0.979807	0.995465	0.978943
5	VGG_Base: Gray	0.995465	0.989474	0.995465	0.989474	0.995505	0.989693	0.995464	0.989474
6	VGG_FFNN	1.000000	0.989474	1.000000	0.989474	1.000000	0.989693	1.000000	0.989474
7	VGG_FFNN: GRAY	1.000000	0.989474	1.000000	0.989474	1.000000	0.989693	1.000000	0.989474
8	VGG_AUG	1.000000	0.978947	1.000000	0.978947	1.000000	0.978947	1.000000	0.978947

show_performance_metrics(history_vgg16_aug_gray, 'VGG_AUG: GRAY', model_vgg_aug_gray, X_train_normalized_gray, y_train_gray, X_val_normalized_gray, y_val_gray)

14/14 ━━━━━━━━━━━━━━━━━━━━ 0s 26ms/step
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 32ms/step
14/14 ━━━━━━━━━━━━━━━━━━━━ 0s 26ms/step
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 33ms/step

Classification Report
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 35ms/step
              precision    recall  f1-score   support

 No Helmet 0       1.00      0.98      0.99        48
    Helmet 1       0.98      1.00      0.99        47

    accuracy                           0.99        95
   macro avg       0.99      0.99      0.99        95
weighted avg       0.99      0.99      0.99        95

	model	Train_Accuracy	Validation_Accuracy	Train_Recall	Validation_Recall	Train_Precission	Validation_Precission	Train_F1	Validation_F1
0	CNN	1.000000	0.978947	1.000000	0.978947	1.000000	0.979807	1.000000	0.978943
1	CNN:Grapy	0.997732	0.989474	0.997732	0.989474	0.997743	0.989693	0.997732	0.989474
2	CNN (Drop=.5)	0.993197	0.978947	0.993197	0.978947	0.993287	0.978947	0.993196	0.978947
3	CNN (Drop=.5): Gray	0.979592	0.989474	0.979592	0.989474	0.979603	0.989693	0.979592	0.989474
4	VGG_Base	0.995465	0.978947	0.995465	0.978947	0.995506	0.979807	0.995465	0.978943
5	VGG_Base: Gray	0.995465	0.989474	0.995465	0.989474	0.995505	0.989693	0.995464	0.989474
6	VGG_FFNN	1.000000	0.989474	1.000000	0.989474	1.000000	0.989693	1.000000	0.989474
7	VGG_FFNN: GRAY	1.000000	0.989474	1.000000	0.989474	1.000000	0.989693	1.000000	0.989474
8	VGG_AUG	1.000000	0.978947	1.000000	0.978947	1.000000	0.978947	1.000000	0.978947
9	VGG_AUG: GRAY	0.997732	0.989474	0.997732	0.989474	0.997743	0.989693	0.997732	0.989474

Visualizing the predictions

show_visulization(X_val, X_val_normalized, y_val, model_vgg_aug, IMAGE_WIDTH, IMAGE_HEIGHT, IMAGE_CHANNEL, index=[2, 10, 23])

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 59ms/step
Predicted Probability: 3.548656968632713e-05, and  Label 0
True Label: 0

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 51ms/step
Predicted Probability: 0.9179984331130981, and  Label 1
True Label: 0

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 55ms/step
Predicted Probability: 0.9999991655349731, and  Label 1
True Label: 1

show_visulization(X_val_gray, X_val_normalized_gray, y_val_gray, model_vgg_aug_gray, IMAGE_WIDTH, IMAGE_HEIGHT, IMAGE_CHANNEL, index=[2, 10, 23])

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 55ms/step
Predicted Probability: 0.00018958865257445723, and  Label 0
True Label: 0

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 55ms/step
Predicted Probability: 0.9983410835266113, and  Label 1
True Label: 0

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 54ms/step
Predicted Probability: 0.9999997615814209, and  Label 1
True Label: 1

Observation

• We freeze VGG 16 convolution layers , add fully connected neural network with 2 hidden layers and feed it to an output layer.
• FNN uses relu activation function and batch normaliztion (he) and adam optimizer.
• As we have small dataset, So, we augment our training dataset varying rotation, using horizontal, and vertical flipping, zooming, etc. The objective is to increase the training datasets with all varieties to avoid overfitting.
• We found that colored image model has very good accuracy, recall, precission, f1 against training almost 100% and in validation, almost 98%.
• We found that gray image model has very good accuracy, recall, precission, f1 against training almost 99.7% and in validation, almost 99%.
• We visualized some images (index 2, 10, 23 randomly) and predicted them. The model correctly predicted them.

Model 5: (VGG-16 (Base + FFNN + Data Augmentation + Dropout)

history_vgg16_aug_drop, model_vgg_aug_drop, history_vgg16_aug_drop_gray, model_vgg_aug_drop_gray = get_vgg_model(only_base=False, train_datagen=train_datagen, drop_out_rate=dropout_rate)

Model: "sequential_24"

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ vgg16 (Functional)              │ (None, 2, 2, 512)      │    14,714,688 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ flatten_24 (Flatten)            │ (None, 2048)           │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_74 (Dense)                │ (None, 256)            │       524,544 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dropout_6 (Dropout)             │ (None, 128)            │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_75 (Dense)                │ (None, 128)            │        32,896 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dropout_6 (Dropout)             │ (None, 128)            │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_76 (Dense)                │ (None, 1)              │           129 │
└─────────────────────────────────┴────────────────────────┴───────────────┘

 Total params: 15,272,257 (58.26 MB)

 Trainable params: 557,569 (2.13 MB)

 Non-trainable params: 14,714,688 (56.13 MB)

Epoch 1/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 177ms/step - accuracy: 0.5716 - loss: 0.8034 - recall: 0.5392 - val_accuracy: 0.9263 - val_loss: 0.2013 - val_recall: 0.8511
Epoch 2/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 37ms/step - accuracy: 0.8125 - loss: 0.3837 - recall: 0.6970 - val_accuracy: 0.9263 - val_loss: 0.1905 - val_recall: 0.8511
Epoch 3/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 162ms/step - accuracy: 0.8900 - loss: 0.2640 - recall: 0.8513 - val_accuracy: 0.9789 - val_loss: 0.0569 - val_recall: 1.0000
Epoch 4/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 36ms/step - accuracy: 0.9062 - loss: 0.2010 - recall: 0.9706 - val_accuracy: 0.9789 - val_loss: 0.0536 - val_recall: 1.0000
Epoch 5/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 169ms/step - accuracy: 0.9434 - loss: 0.1342 - recall: 0.9576 - val_accuracy: 0.9789 - val_loss: 0.0475 - val_recall: 0.9574
Epoch 6/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 36ms/step - accuracy: 0.9844 - loss: 0.0662 - recall: 1.0000 - val_accuracy: 0.9789 - val_loss: 0.0491 - val_recall: 0.9574
Epoch 7/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 174ms/step - accuracy: 0.9659 - loss: 0.1128 - recall: 0.9525 - val_accuracy: 0.9789 - val_loss: 0.0348 - val_recall: 0.9787
Epoch 8/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 37ms/step - accuracy: 0.9531 - loss: 0.0815 - recall: 0.9429 - val_accuracy: 0.9789 - val_loss: 0.0342 - val_recall: 0.9787
Epoch 9/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 170ms/step - accuracy: 0.9682 - loss: 0.0953 - recall: 0.9683 - val_accuracy: 0.9895 - val_loss: 0.0381 - val_recall: 0.9787
Epoch 10/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 38ms/step - accuracy: 1.0000 - loss: 0.0131 - recall: 1.0000 - val_accuracy: 0.9895 - val_loss: 0.0386 - val_recall: 0.9787

Model: "sequential_25"

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ vgg16 (Functional)              │ (None, 2, 2, 512)      │    14,714,688 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ flatten_25 (Flatten)            │ (None, 2048)           │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_77 (Dense)                │ (None, 256)            │       524,544 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dropout_7 (Dropout)             │ (None, 128)            │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_78 (Dense)                │ (None, 128)            │        32,896 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dropout_7 (Dropout)             │ (None, 128)            │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_79 (Dense)                │ (None, 1)              │           129 │
└─────────────────────────────────┴────────────────────────┴───────────────┘

 Total params: 15,272,257 (58.26 MB)

 Trainable params: 557,569 (2.13 MB)

 Non-trainable params: 14,714,688 (56.13 MB)

Epoch 1/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 178ms/step - accuracy: 0.6258 - loss: 0.7246 - recall: 0.5161 - val_accuracy: 0.9684 - val_loss: 0.1155 - val_recall: 0.9787
Epoch 2/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 36ms/step - accuracy: 0.8281 - loss: 0.3493 - recall: 0.7419 - val_accuracy: 0.9684 - val_loss: 0.0946 - val_recall: 0.9787
Epoch 3/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 161ms/step - accuracy: 0.8802 - loss: 0.2530 - recall: 0.8991 - val_accuracy: 0.9789 - val_loss: 0.0519 - val_recall: 0.9787
Epoch 4/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 37ms/step - accuracy: 0.9219 - loss: 0.2192 - recall: 0.8824 - val_accuracy: 0.9789 - val_loss: 0.0494 - val_recall: 0.9787
Epoch 5/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 166ms/step - accuracy: 0.9596 - loss: 0.1207 - recall: 0.9836 - val_accuracy: 0.9895 - val_loss: 0.0449 - val_recall: 1.0000
Epoch 6/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 37ms/step - accuracy: 0.9688 - loss: 0.0672 - recall: 0.9667 - val_accuracy: 0.9895 - val_loss: 0.0451 - val_recall: 1.0000
Epoch 7/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 166ms/step - accuracy: 0.9843 - loss: 0.0591 - recall: 0.9786 - val_accuracy: 0.9895 - val_loss: 0.0489 - val_recall: 1.0000
Epoch 8/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 36ms/step - accuracy: 0.9688 - loss: 0.0839 - recall: 0.9714 - val_accuracy: 0.9895 - val_loss: 0.0466 - val_recall: 1.0000
Epoch 9/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 1s 170ms/step - accuracy: 0.9836 - loss: 0.0696 - recall: 0.9819 - val_accuracy: 0.9895 - val_loss: 0.0408 - val_recall: 1.0000
Epoch 10/10
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 37ms/step - accuracy: 0.9649 - loss: 0.0771 - recall: 0.9677 - val_accuracy: 0.9895 - val_loss: 0.0436 - val_recall: 1.0000

show_performance_metrics(history_vgg16_aug_drop, 'VGG16_AUG (Drop=.5)', model_vgg_aug_drop, X_train_normalized, y_train, X_val_normalized, y_val)

14/14 ━━━━━━━━━━━━━━━━━━━━ 0s 26ms/step
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 34ms/step
14/14 ━━━━━━━━━━━━━━━━━━━━ 0s 27ms/step
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 34ms/step

Classification Report
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 34ms/step
              precision    recall  f1-score   support

 No Helmet 0       0.98      1.00      0.99        48
    Helmet 1       1.00      0.98      0.99        47

    accuracy                           0.99        95
   macro avg       0.99      0.99      0.99        95
weighted avg       0.99      0.99      0.99        95

	model	Train_Accuracy	Validation_Accuracy	Train_Recall	Validation_Recall	Train_Precission	Validation_Precission	Train_F1	Validation_F1
0	CNN	1.000000	0.978947	1.000000	0.978947	1.000000	0.979807	1.000000	0.978943
1	CNN:Grapy	0.997732	0.989474	0.997732	0.989474	0.997743	0.989693	0.997732	0.989474
2	CNN (Drop=.5)	0.993197	0.978947	0.993197	0.978947	0.993287	0.978947	0.993196	0.978947
3	CNN (Drop=.5): Gray	0.979592	0.989474	0.979592	0.989474	0.979603	0.989693	0.979592	0.989474
4	VGG_Base	0.995465	0.978947	0.995465	0.978947	0.995506	0.979807	0.995465	0.978943
5	VGG_Base: Gray	0.995465	0.989474	0.995465	0.989474	0.995505	0.989693	0.995464	0.989474
6	VGG_FFNN	1.000000	0.989474	1.000000	0.989474	1.000000	0.989693	1.000000	0.989474
7	VGG_FFNN: GRAY	1.000000	0.989474	1.000000	0.989474	1.000000	0.989693	1.000000	0.989474
8	VGG_AUG	1.000000	0.978947	1.000000	0.978947	1.000000	0.978947	1.000000	0.978947
9	VGG_AUG: GRAY	0.997732	0.989474	0.997732	0.989474	0.997743	0.989693	0.997732	0.989474
10	VGG16_AUG (Drop=.5)	0.997732	0.989474	0.997732	0.989474	0.997743	0.989689	0.997732	0.989471

show_performance_metrics(history_vgg16_aug_drop_gray, 'VGG16_AUG (Drop=.5): GRAY', model_vgg_aug_drop_gray, X_train_normalized_gray, y_train_gray, X_val_normalized_gray, y_val_gray)

14/14 ━━━━━━━━━━━━━━━━━━━━ 0s 27ms/step
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 34ms/step
14/14 ━━━━━━━━━━━━━━━━━━━━ 0s 27ms/step
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 34ms/step

Classification Report
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 38ms/step
              precision    recall  f1-score   support

 No Helmet 0       1.00      0.98      0.99        48
    Helmet 1       0.98      1.00      0.99        47

    accuracy                           0.99        95
   macro avg       0.99      0.99      0.99        95
weighted avg       0.99      0.99      0.99        95

	model	Train_Accuracy	Validation_Accuracy	Train_Recall	Validation_Recall	Train_Precission	Validation_Precission	Train_F1	Validation_F1
0	CNN	1.000000	0.978947	1.000000	0.978947	1.000000	0.979807	1.000000	0.978943
1	CNN:Grapy	0.997732	0.989474	0.997732	0.989474	0.997743	0.989693	0.997732	0.989474
2	CNN (Drop=.5)	0.993197	0.978947	0.993197	0.978947	0.993287	0.978947	0.993196	0.978947
3	CNN (Drop=.5): Gray	0.979592	0.989474	0.979592	0.989474	0.979603	0.989693	0.979592	0.989474
4	VGG_Base	0.995465	0.978947	0.995465	0.978947	0.995506	0.979807	0.995465	0.978943
5	VGG_Base: Gray	0.995465	0.989474	0.995465	0.989474	0.995505	0.989693	0.995464	0.989474
6	VGG_FFNN	1.000000	0.989474	1.000000	0.989474	1.000000	0.989693	1.000000	0.989474
7	VGG_FFNN: GRAY	1.000000	0.989474	1.000000	0.989474	1.000000	0.989693	1.000000	0.989474
8	VGG_AUG	1.000000	0.978947	1.000000	0.978947	1.000000	0.978947	1.000000	0.978947
9	VGG_AUG: GRAY	0.997732	0.989474	0.997732	0.989474	0.997743	0.989693	0.997732	0.989474
10	VGG16_AUG (Drop=.5)	0.997732	0.989474	0.997732	0.989474	0.997743	0.989689	0.997732	0.989471
11	VGG16_AUG (Drop=.5): GRAY	0.997732	0.989474	0.997732	0.989474	0.997743	0.989693	0.997732	0.989474

Visualizing the predictions

show_visulization(X_val, X_val_normalized, y_val, model_vgg_aug_drop, IMAGE_WIDTH, IMAGE_HEIGHT, IMAGE_CHANNEL, index=[2, 10, 23])

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 56ms/step
Predicted Probability: 0.00039803964318707585, and  Label 0
True Label: 0

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 57ms/step
Predicted Probability: 0.47181960940361023, and  Label 0
True Label: 0

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 58ms/step
Predicted Probability: 0.9999116659164429, and  Label 1
True Label: 1

show_visulization(X_val_gray, X_val_normalized_gray, y_val_gray, model_vgg_aug_drop_gray, IMAGE_WIDTH, IMAGE_HEIGHT, IMAGE_CHANNEL, index=[2, 10, 23])

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 58ms/step
Predicted Probability: 0.0005008371663279831, and  Label 0
True Label: 0

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 56ms/step
Predicted Probability: 0.9678416848182678, and  Label 1
True Label: 0

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 57ms/step
Predicted Probability: 0.9999979734420776, and  Label 1
True Label: 1

• We freeze VGG 16 convolution layers , add fully connected neural network with 2 hidden layers and feed it to an output layer.
• FNN uses relu activation function and batch normaliztion (he) and adam optimizer.
• As we have small dataset, So, we augment our training dataset varying rotation, using horizontal, and vertical flipping, zooming, etc. The objective is to increase the training datasets with all varieties to avoid overfitting.
• We used 50% drop out to avoid overfitting
• We found that both color and gray imge models have very good accuracy, recall, precission, and f1 against training, is almost 99.7% and validation, almost 99%.
• We visualized some images (index 2, 10, 23 randomly) and predicted them. The model correctly predicted them.

Model Performance Comparison and Final Model Selection

results_eval

	model	Train_Accuracy	Validation_Accuracy	Train_Recall	Validation_Recall	Train_Precission	Validation_Precission	Train_F1	Validation_F1
0	CNN	1.000000	0.978947	1.000000	0.978947	1.000000	0.979807	1.000000	0.978943
1	CNN:Grapy	0.997732	0.989474	0.997732	0.989474	0.997743	0.989693	0.997732	0.989474
2	CNN (Drop=.5)	0.993197	0.978947	0.993197	0.978947	0.993287	0.978947	0.993196	0.978947
3	CNN (Drop=.5): Gray	0.979592	0.989474	0.979592	0.989474	0.979603	0.989693	0.979592	0.989474
4	VGG_Base	0.995465	0.978947	0.995465	0.978947	0.995506	0.979807	0.995465	0.978943
5	VGG_Base: Gray	0.995465	0.989474	0.995465	0.989474	0.995505	0.989693	0.995464	0.989474
6	VGG_FFNN	1.000000	0.989474	1.000000	0.989474	1.000000	0.989693	1.000000	0.989474
7	VGG_FFNN: GRAY	1.000000	0.989474	1.000000	0.989474	1.000000	0.989693	1.000000	0.989474
8	VGG_AUG	1.000000	0.978947	1.000000	0.978947	1.000000	0.978947	1.000000	0.978947
9	VGG_AUG: GRAY	0.997732	0.989474	0.997732	0.989474	0.997743	0.989693	0.997732	0.989474
10	VGG16_AUG (Drop=.5)	0.997732	0.989474	0.997732	0.989474	0.997743	0.989689	0.997732	0.989471
11	VGG16_AUG (Drop=.5): GRAY	0.997732	0.989474	0.997732	0.989474	0.997743	0.989693	0.997732	0.989474

Observation

• All the models have almost 97-100% accuracy, recall, precission and F1 score both in training and testing.
• This is because of size of dataset and I believe, dataset is very clean and proccessed.
• Upon all of the above, I will choose VGG16 with augmentation and drop out of 50% as it will give stable or resilient prediction performance (theoretically) in unseen environment with realtime work monitoring. It's recall and f1 score in training is almost 99.5% but in testing 98% (very close and not 100%). Moreover, Model built with augmented training data learns varities of images with different positioning, and dropping out in hidden layer with help to overcome overfitting.
• Although VGG-16 augmentation with 50% drop out models for both color and gray images have the same result, we will pick the color image model as it trains will all color channels and can predict color images without converting to gray scale.

Test Performance

columns_eval_metric = ["model", "Train_Accuracy","Testing_Accuracy", "Train_Recall", "Testing_Recall", "Train_Precission", "Testing_Precission", "Train_F1","Testing_F1"]

results_eval = pd.DataFrame(columns=columns_eval_metric)
show_performance_metrics(history_vgg16_aug_drop, 'VGG16_AUG (Drop=.5)', model_vgg_aug_drop, X_train_normalized, y_train, X_test_normalized, y_test, is_testing = True)

14/14 ━━━━━━━━━━━━━━━━━━━━ 0s 29ms/step
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 34ms/step
14/14 ━━━━━━━━━━━━━━━━━━━━ 0s 27ms/step
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 33ms/step

Classification Report
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 34ms/step
              precision    recall  f1-score   support

 No Helmet 0       1.00      1.00      1.00        48
    Helmet 1       1.00      1.00      1.00        47

    accuracy                           1.00        95
   macro avg       1.00      1.00      1.00        95
weighted avg       1.00      1.00      1.00        95

	model	Train_Accuracy	Testing_Accuracy	Train_Recall	Testing_Recall	Train_Precission	Testing_Precission	Train_F1	Testing_F1
0	VGG16_AUG (Drop=.5)	0.997732	1.0	0.997732	1.0	0.997743	1.0	0.997732	1.0

Observation

• VGG augmented color image model with drop out of 50% has also good testing evaluation metrics of 100%

Actionable Insights & Recommendations

Actionable Insight

• We build models using colored and gray scaled images. Hence, we have 12 models.

• We classified images of human with safe helmet in their working place. We were given a dataset of 631 images and 320 of them are without helmet and 311 are with helmet.

• The dataset is very small and uniformly distributed. Therefore, we have amost 97-100% accuracy by all models (colored and gray scaled) of training, validation, and testing performance and have overfitting.

• We observe that, basic CNN/CNN with drop out (50%) has 97% validation recall where as all VGG-16 models (basic, FFNN, AUG, and AUG with drop out of 50%) have almost 99% validation records. This proofs that VGG-16 pre-trained features were beneficial, further refining them with an additional neural network tailored to different dataset enhanced the results.

• However, all the VGG-16-based models are experiencing overfitting. To address this, data augmentation techniques and drop out of 50% can be applied as a form of regularization with the additional varieties for datasets in different work places.

• We choose VGG augmented colored image model with drop out of 50% as to overcome overfitting. It has also good testing evaluation metrics of 100%

Recommendation

• Business can integrate this model into their real-time workplace safety monitoring system, and incrementally improve model by feeding more data.

• These models can be further improved by training with different filter sizes and different number of filters.

• These models can also be trained on the original image_size i.e 200 x 200 rather than being reduced to 64.

• Data Augmentation can be performed more and dropout_rate can be changed to improve the model performance.

• Other Transfer Learning architectures can also be used to train the CNN model and these models can be used for classification.

Power Ahead!

---